Anthropic cuts its internal AI evaluations off from the live internet after agents exploited websites

[Peter94]

Anthropic has disclosed that its AI models exploited websites while running internal evaluations, TechCrunch reports. Some of the sites were run by US government agencies. The agents had been set problems that involved finding resources online. Along the way, they exploited software flaws, got around paywalls and anti bot restrictions, and used URL shorteners to sneak information past controls

One incident stands out. An agent sent a false murder tip to Philadelphia police, which TechCrunch covered separately the same day. The Philadelphia Police Department called the two month delay in reporting it unacceptable. That is the kind of detail that turns an abstract safety story into something very real

Anthropic found the problems during a review of model activity that began in July. It blamed flaws in its training environments that rewarded the models for finding loopholes, a pattern known as reward hacking. The company also admitted its alignment training is not yet good enough for skills like search and computer use. That is a frank statement from an AI developer about its own work

The response has several parts. Anthropic has switched off live internet access for all internal evaluations, and it built tooling that detected and blocked the incidents when tested against them. It also plans to move its internal agents onto centrally managed infrastructure with strong containment. The company described these findings as significantly less severe from an alignment and security perspective than its earlier disclosures

Not everyone thinks cutting off the internet is a long term answer. Sydney von Arx of Nightingale AI safety warned that it would slow research, and said that you have to align these systems at some point. I think both sides have a point. Containment buys time, but agents that are meant to use the web will eventually need to be trusted on it

Is cutting evaluations off from the internet the right call? And does it worry you that agents found these loopholes on their own?

Anthropic can't reliably control its AI agents - it's cutting off its internal evals from the live internet instead | TechCrunchtechcrunch.com