AI's version of harvest now decrypt later, stealing model weights before you can even use them

[Gold Terry]

A different kind of theft than most people picture

When people worry about AI security breaches, they usually picture data leaks, jailbreaks, or someone tricking a chatbot into saying something embarrassing. The scenario security researchers actually lose sleep over is narrower and stranger than any of that, someone stealing the model weights themselves, the actual learned numerical parameters that took hundreds of millions of dollars and months of dedicated computation to produce in the first place. Gaining access to a frontier model's weights is functionally equivalent to having trained that model yourself, without paying any of the enormous cost involved, and closed frontier models are on average meaningfully more capable than anything currently available openly, which makes them an unusually attractive target for anyone ranging from opportunistic cybercrime syndicates all the way up to well resourced nation states

Why this is genuinely different from ordinary corporate espionage

A stolen model arrives with none of the guardrails a legitimate developer would normally deploy alongside it in production. Safety fine tuning, usage monitoring, refusal behaviors built in for dangerous requests, all of that can be stripped away entirely by a sufficiently sophisticated actor working directly with the raw weights, completely outside of any developer's oversight or control. That means the exact same underlying capability that a legitimate company deploys carefully and cautiously becomes available to someone facing none of the reputational, legal, or commercial pressure that currently keeps frontier AI labs behaving cautiously in the first place

As of the most recent comprehensive assessment contained in the International AI Safety Report, there are no confirmed, publicly documented cases of a frontier model's weights actually being stolen to date, though other real security breaches at major AI companies, including a documented infiltration of Microsoft's own email systems, have already been reported and confirmed. That absence of any confirmed case is genuinely reassuring on one level and genuinely worrying on another simultaneously, since RAND's own dedicated research into this exact question concluded that current security practices across the industry as a whole remain insufficient, and that even substantial additional investment in security may still not be enough to reliably defend against the very most well resourced state level attackers specifically

The harvest now, decrypt later parallel

This is where the comparison to quantum computing's most widely discussed cryptographic threat becomes genuinely useful rather than just a convenient analogy. In that world, adversaries are already recording encrypted data today, betting that a future sufficiently powerful quantum computer will eventually let them decrypt it retroactively at some later date. Something structurally quite similar could apply here, a model that gets stolen today but is not yet fully understood or exploitable by the thief might become far more dangerous later, once that attacker develops the specific fine tuning expertise needed, or eventually discovers some specific dangerous capability latent inside those particular weights that was not obvious at the time of the theft itself. The theft and the actual exploitation do not have to happen at the same time at all, which is exactly what makes this genuinely hard to defend against using standard incident response thinking built primarily around detecting and stopping an attack as it happens in real time

The five level framework the industry is unevenly working toward

Security researchers have proposed a five tier maturity model describing how seriously different labs are actually protecting their weights in practice, with the top tier, Security Level 5, requiring defenses genuinely capable of plausibly thwarting attack vectors available to the most highly resourced state actors in the world, keeping weights physically air gapped inside something equivalent to a Sensitive Compartmented Information Facility, the exact kind of security normally reserved specifically for classified government material. Reaching that level requires successfully defending against roughly forty distinct attack vectors that researchers have already carefully catalogued, spanning everything from compromised access controls and physical intrusion attempts to supply chain attacks and human insider threats, and current industry practice varies enormously from lab to lab, some developers have made detailed public security commitments specifically addressing this exact threat, while others have made essentially none at all

The asymmetry that makes this uniquely hard to fix

Unlike a lot of other AI safety problems currently being discussed, this one is not really a technical alignment challenge at its core, it is a conventional, if extremely high stakes, cybersecurity problem, and conventional cybersecurity problems come with a brutal built in asymmetry, a defender has to succeed against literally every single attempt indefinitely into the future, while an attacker only ever has to succeed once. For a technology this valuable and this consequential, that asymmetry alone is reason enough for weight security to deserve exactly the same level of institutional seriousness the industry already gives to model alignment research, even though it currently receives only a small fraction of the equivalent public attention and discourse

Sources

International AI Safety Report 2026arxiv.org

arXiv, AI emergency preparedness and weight theft scenariosarxiv.org

arXiv, generative AI in cybersecurityarxiv.org

arXiv, approval regulation and security controls for frontier AI