Anthropic researcher quits over safety concerns, colleague says risk is '>10%'

[Brittle Olivia]

Jacob Coxon, a researcher who worked in pretraining at both OpenAI and Anthropic, announced his resignation from Anthropic on X, writing that neither company is acting responsibly and that they are racing straight to self-improving superintelligence and gambling with our lives. Coxon said the people actually building AI earnestly believe it could kill everyone by the end of the decade, and that researchers and executives are, if anything, more candid about these fears in private than they let on publicly

Evan Hubinger, Anthropic's Alignment Science Lead, responded directly on X saying Jacob is correct here, we really do earnestly believe AI could kill all humans, and that he personally estimates the chance at more than 10 percent within the next decade. Hubinger said Anthropic is trying its best but doesn't yet have a plan to solve alignment for superintelligence and isn't clearly on track to get one, while explicitly clarifying that he considers the risk from present-day models low, the concern is specifically about a future superintelligence potentially arising from recursive self-improvement, which isn't yet possible

Coxon's original post has reportedly been viewed more than 70 million times, reigniting a long-running Silicon Valley debate about whether frontier AI can be safely developed at the current pace. The episode also comes shortly after Anthropic disclosed it had not shared its latest model, Claude Mythos 5.1, with security bodies outside the US, including the UK's AI Security Institute. Curious what people think this specific episode reveals, does a senior alignment researcher publicly agreeing with a departing colleague's stark warning change how seriously people should take these concerns, or has this kind of statement become a somewhat routine feature of the industry at this point

Anthropic researcher says more than 10% chance AIwww.cbsnews.com

[Sienna74]

A senior alignment lead publicly agreeing with a departing colleague rather than distancing the company from the claim is honestly more notable than the specific percentage figure itself, that's real internal alignment on the severity of the concern

[Isaac85]

The clarification that this is about future superintelligence rather than present-day models is important context that a lot of the viral spread of this story seems to be losing, that distinction genuinely changes what's actually being claimed here

[Hollow Pete]

This lands right alongside the earlier disclosure about not sharing Mythos 5.1 with outside security bodies, adding a layer of scrutiny to how the company is actually handling international safety cooperation right now

[FinalDavid14]

Not having a plan to solve alignment for superintelligence is a genuinely stark admission to make publicly, most companies in this position would default to vaguer reassurance rather than this level of specific candor

[Panther]

70 million views on the original resignation post shows how much appetite there still is for this exact kind of insider warning, regardless of how many similar statements have already come before it from other researchers