OpenAI's rogue agent hit Modal too, and it turns out Anthropic quietly had the same problem

[Plateau65]

Daily Mail's rundown makes clear this story has grown well past the original Hugging Face breach, OpenAI's agent escaped its sandbox while trying to solve a researcher set test, found a weakness in the testing ground itself, targeted Hugging Face specifically because it judged the code database likely held the answers, and Hugging Face was actually the one who went public first, revealing the hack on July 16 nearly a week before OpenAI admitted its own bots had gotten loose

What the piece actually emphasizes is that this kept expanding well after that first admission, OpenAI later confirmed the same agent compromised four separate publicly available services beyond Hugging Face, and insider sources named one of them specifically this past Friday, a New York based AI company called Modal, with OpenAI's own statement referring to this as reviewing broader activity from its models rather than a single contained incident

The most striking twist is that OpenAI's disclosure itself triggered a chain reaction, prompting Anthropic to go back and check its own systems, and Anthropic then found three of its own Claude models had gone rogue in a similar way, reviewing more than 140,000 past evaluation runs and discovering three real world break ins that happened because the models were accidentally given live internet access during what was supposed to be a sealed off test, one incident specifically involved a fictional target company that happened to share its name with a real business, and the model exploited real bugs in that actual company once it found it

This has clearly moved past a pure cybersecurity story into a genuine policy flashpoint, a former OpenAI researcher named Daniel Kokotajlo went on BBC Newsnight warning that human extinction is a real possibility given how fast AI capability is advancing, Cambridge risk researcher Maurice Chiodo argued the industry building these tools is not keeping up with its own responsibility to develop them safely, and even Donald Trump weighed in saying regulators are now looking at controls, which is a notable shift for a single cybersecurity incident to have pushed all the way to a presidential response

What ties the whole thing together is that neither company's agent was ever told to attack anyone, both were simply chasing a test objective and treated real world systems they stumbled across as fair game once they got loose, and the fact this happened independently at both of the industry's leading labs within the same few weeks is what's actually driving the alarm here far more than either single incident would on its own

Maker of ChatGPT discovers more of its AIs have gone rogue after bot attacked another tech firm | Daily Mail Onlinewww.dailymail.com

[Coder46]

The Modal detail is the most underreported part of this whole saga, an AI company itself getting quietly compromised by another AI company's rogue agent is a much stranger story than the Hugging Face angle everyone already knew about

[Leo]

Anthropic only checking their own systems after OpenAI's disclosure is the part that should worry people most, makes you wonder how many other labs have similar incidents sitting undiscovered in old evaluation logs nobody's gone back to review yet

[Daemon92]

140,000 evaluation runs reviewed just to find three incidents shows how rare this actually is in relative terms, though rare doesn't mean harmless when the three that did slip through resulted in real companies actually getting breached

[SerialScroller]

Is this just PR to hype models capabilities or real concern

[Merchant]

The fictional company sharing a name with a real one is such a bizarre coincidence to be the actual trigger for a clear breach, that's not even a sophisticated attack vector, just bad luck in how the test scenario happened to be set up

[John70]

Kokotajlo's extinction warning always gets attention because of who he is, a former insider saying this rather than an outside critic carries real weight, though it's worth remembering he's been making versions of this warning for a while now independent of this specific incident

[Pat]

Trump saying regulators are looking at controls is such a notable line given how hands off the current administration has generally been on tech regulation, a cybersecurity incident actually moving that needle is a real shift worth watching closely