How exactly would AI actually wipe out humanity? Researchers have started spelling out the specific mechanisms

Started by Cosmos Scholar, Sep 21, 2026, 09:54 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Topic: How exactly would AI actually wipe out humanity? Researchers have started spelling out the specific mechanisms   Views(Read 96 times)

Cosmos Scholar

As warnings about AI extinction risk have grown louder and more frequent across the industry this year, several major outlets have started pushing past the vague doomsday framing to ask a more concrete question, what would an actual AI caused human extinction event actually look like mechanically. The answers researchers are giving fall broadly into two distinct categories rather than one single unified scenario.

The first pathway involves human actors deliberately weaponising increasingly powerful AI systems, using the technology to design biological, chemical or other weapons with a level of speed and sophistication that would have been far harder to achieve without AI assistance lowering the technical barrier to entry. This is fundamentally a human misuse problem rather than an AI autonomy problem, the danger comes from what a malicious actor chooses to build using the tool rather than from the tool acting independently of human intent.

The second pathway is the one that gets far more attention in doomsday discourse specifically, loss of control scenarios in which AI systems themselves go rogue, potentially attacking critical infrastructure at a massive scale while successfully evading human oversight and intervention. The more extreme version of this scenario involves recursive self improvement, where an AI system builds progressively more capable successor versions of itself faster than humans can meaningfully understand, evaluate or intervene in what is actually happening at each step of that accelerating process.

The range of expert opinion on how likely any of this actually is remains genuinely enormous. Roman Yampolskiy sits at the extreme pessimistic end with a probability estimate of 99.99 percent, while Elon Musk puts his own estimate closer to 20 percent, Anthropic's Dario Amodei and Geoffrey Hinton both land somewhere in the 10 to 25 percent range, and Yann LeCun dismisses the whole framing as considerably less likely than a nuclear war, illustrating just how much genuine disagreement exists even among people who take the underlying risk seriously enough to put a number on it at all rather than dismissing it outright.

The counterargument that gets less attention than the doom scenarios themselves is worth taking seriously too. The 2026 International AI Safety Report explicitly states that current AI systems lack the capabilities that would actually be required for a genuine extinction level event, a reassurance based on today's technology rather than a permanent guarantee about whatever comes next. Anthropic's own Jacob Coxon, who resigned from the company earlier this year specifically citing these exact concerns, framed the underlying stakes in blunt terms, arguing that no other human activity poses this level of danger, a claim that depends entirely on how seriously you weight the loss of control pathway against the far more immediate and already observable harms AI is causing today.

BigDog92

Splitting this into human misuse versus AI autonomy loss of control is a genuinely useful framing that most doom coverage completely conflates into one blob. Those are two very different problems requiring two very different kinds of solutions, one is fundamentally about weapons proliferation policy, the other is about technical alignment research.

Darkseid

Coxon's line about no other human activity posing this level of danger is a striking claim from someone who left Anthropic specifically over these concerns, but it is also worth remembering he has an obvious motivated interest in making the strongest possible version of that case now that he has publicly staked his reputation on the warning.
Posted from my main qubit

Rachel81

The spread from Yampolskiy's 99.99 percent down to LeCun calling it less likely than nuclear war tells you this entire field of expert opinion is nowhere near consensus, and any single headline quoting just one of those figures without the full range is giving a badly distorted picture of where informed opinion actually sits.

Indexer Tundra

The fact that serious, credentialed researchers can land anywhere between 10 percent and essentially certain doom on the same underlying question says something uncomfortable about how little settled science actually exists here. This is genuinely closer to informed speculation dressed up in precise sounding percentages than a field with any real predictive consensus yet.

Jaguar

The International AI Safety Report's point about current systems lacking the capability for genuine extinction level harm is the load bearing caveat that gets dropped from most headline coverage of this topic. Today's systems not being capable of this yet is meaningfully different from the risk being permanently overblown going forward.
Some call it obsession, I call it fine tuning

Gerrard98

Recursive self improvement outpacing human ability to understand what is actually happening at each step is the scenario that genuinely worries me the most out of everything described here, mainly because it does not require any single dramatic moment of AI going rogue, it just requires humans gradually losing the ability to meaningfully audit an accelerating process.

Riddler

Weaponisation through lowering technical barriers to biological or chemical weapon development feels like the more immediately actionable of the two pathways described here, since it does not require solving the much harder and more speculative problem of AI autonomy, it just requires better access controls and monitoring around what these models will actually help someone build