Daz92

A single number carrying a lot of regulatory weight

Most current AI regulation, the EU AI Act and the framework that briefly existed under the 2023 US executive order before it was later revoked, both draw their core dividing line using the exact same tool, a raw count of floating point operations, or FLOPs, used to train a given model. Cross a threshold, currently 10 to the 25th power FLOPs in the EU and originally 10 to the 26th power in the US, and your model gets formally classified as posing systemic risk, triggering mandatory adversarial testing, incident reporting to regulators, and formal cybersecurity assessments. Stay under that number, and none of those obligations apply to you at all

The appeal of this approach is obvious, it gives regulators one clean, quantifiable, genuinely hard to fake number to point to, rather than forcing them into a subjective, contestable judgment call about how dangerous any particular model actually is in practice. The problem is that the number and the actual underlying danger are already coming apart from each other, and the gap appears to be widening rather than closing

The DeepSeek problem

DeepSeek's R1 model became the clearest public proof of concept for why this entire framework is shakier than it first appears. Through more efficient architecture and smarter training techniques, R1 achieved performance broadly comparable to models from leading Western labs while using dramatically less raw compute in the process, directly undermining the core underlying assumption that FLOPs reliably predict capability in the first place. If a sufficiently clever team can get a frontier level model to sit safely under the regulatory threshold purely through better engineering rather than through any actual reduction in real world capability, the threshold stops functioning as a meaningful dividing line between dangerous and safe models entirely

The problem cuts in the opposite direction too, and arguably just as severely. Research into inference scaling has shown that a model trained with only modest compute can have its effective real world capability amplified enormously simply by throwing more computing power at it during deployment rather than during the original training run, meaning a model that looks safely compliant and under the threshold on paper could still be scaled up in practice to perform like a model several orders of magnitude above it, purely by changing how it gets used after release rather than how it was originally built. That breaks the entire underlying premise of regulating the object, the trained model itself, rather than the use, since the exact same underlying model could end up compliant or non compliant depending entirely on how much inference compute someone chooses to throw at it after the fact, well outside the original developer's control

The static threshold problem

Even setting efficiency gains aside entirely, frontier training compute has been roughly doubling every six months, a staggering 350 million fold increase over about 13 years, while the regulatory thresholds themselves remain static numbers written directly into law, only changeable through a formal legislative or European Commission review process. The EU AI Act does technically allow the Commission to adjust its own threshold over time to reflect ongoing technological change, but it notably does not specify any automatic formula for actually doing so, meaning the number sits at permanent risk of quietly drifting out of step with the technology it claims to regulate, unless someone actively intervenes at the political level to update it

Researchers have proposed several patches to this design flaw, replacing fixed numerical thresholds with adaptive ones defined relative to the current capability frontier rather than pinned to an absolute static number, or layering direct capability based evaluations on top of the existing compute threshold as a second, considerably harder to game check, rather than relying on FLOPs as the sole gatekeeping metric. Other researchers argue the entire approach is fundamentally misconceived from the start, and that governance should shift decisively toward evaluating actual demonstrated capabilities and genuine real world harms directly, rather than continuing to lean on a training time proxy that was always going to be, at best, an imperfect stand in for the thing regulators actually care about

Why this matters well beyond wonky policy debates

This connects directly back to the software only singularity question discussed elsewhere on this board. If algorithmic efficiency keeps improving fast enough, and if a meaningful fraction of genuinely dangerous capability gains can come from smarter software and better use of existing compute rather than from bigger, more expensive training runs, then compute thresholds are attempting to regulate exactly the wrong underlying variable from the very start. The EU AI Act becomes fully enforceable in August 2026, making this a live, immediate governance question rather than an abstract academic one, and it is genuinely plausible that the flagship regulatory framework built specifically to govern frontier AI is already measuring the wrong thing on the very day it takes full legal effect across the entire bloc




arXiv, inference scaling and governance implications
First post best post ever!!

NWO

DeepSeek R1 is such a perfect real world case study for this whole argument, it's not a hypothetical anymore, it actually happened and it actually undermined the core premise in a way regulators clearly hadn't fully planned for
I read every reply. Even the bad ones.

Transformer Curtis

The inference scaling point is the one that really breaks my brain a bit, the same exact trained model being compliant or non compliant purely based on how much compute someone throws at it after release feels like a fundamentally unfixable design flaw baked into the whole approach
git commit -m "fixed everything"

Context Sookie

350 million fold increase in compute over 13 years sitting next to a threshold that only updates through slow formal legislative review is such a mismatch in timescales, genuinely no wonder it's already lagging behind reality
My team is always one signing away

Cobra69

Regulating the object instead of the use case is the core design mistake running through all of this, most other genuinely risky technologies get regulated based on what you actually do with them, not just how they happened to be manufactured

Static Estuary

Adaptive thresholds pegged to the current capability frontier instead of a fixed absolute number seems like such an obviously better design in hindsight, curious why that wasn't just the default choice from the very start
git commit -m "fixed everything"

TheRizz00

August 2026 full enforceability landing right in the middle of all this unresolved uncertainty about whether the core threshold even means anything anymore is genuinely unfortunate timing for the EU specifically

Craig90

This is a good reminder that writing genuinely good policy for a fast moving technology is hard in a real, structural sense, not just a matter of politicians not caring enough or not trying hard enough

Quarry18

That mismatch between fast-moving compute trends and slow-moving regulation is hard to ignore. Laws are designed for stability, but AI development is anything but stable.

By the time thresholds are debated and approved, the underlying reality has already shifted forward.
Have you tried turning it off and on again?

QuantumLeap11

There is also the question of whether fixed thresholds are the right approach at all.

Tying regulation to a number that quickly becomes outdated seems fragile.

More adaptive frameworks might make more sense, though they come with their own challenges.

ForumGremlin

Part of the issue is that legislation needs clarity, while technology evolves through ambiguity.

That tension makes it difficult to write rules that remain relevant for long.

What looks precise on paper can age very quickly :-\
Gunners for life.

Skibidi98

Some would argue that even imperfect regulation is better than none.

It sets a baseline and signals intent, even if it needs updating later.

The real problem is how quickly those updates can happen.

Jude_54

There is an interesting comparison to financial regulation.

Markets move fast too, but regulators often rely on principles rather than fixed thresholds.

That flexibility might be worth borrowing here.

Romulan32

The compute metric itself might not capture everything that matters.

Efficiency gains, algorithmic improvements, and data quality all play a role.

Focusing on one dimension risks missing the bigger picture.

QuantumToken57

At the same time, expecting regulation to perfectly track innovation might be unrealistic.

A certain level of lag is probably inevitable.

The goal might be to minimize the gap rather than eliminate it entirely :)

Shannon

The speed issue is a real challenge because legislation is naturally slower than technology, but that does not mean regulation is pointless. If every rule waited until the technology stopped changing, we would never get anything written at all.

The trick seems to be building systems that can adapt without needing a full political battle every time a new model gets released. Easier said than done though :)

Rocket67

The funny thing about AI regulation is that people complain both when governments move too slowly and when they move too quickly. Somehow regulators are expected to predict the future while also not making any mistakes along the way.

A flexible framework with regular updates might work better than trying to lock everything into a single definition of what AI risk looks like.

DeepInlet

Thresholds based on compute probably make sense as a rough signal, but they are not the whole story. A smaller model with poor deployment choices could still cause problems, while a huge model might be used responsibly.

Regulation needs to look at capability, access, and actual impact rather than just one technical measurement.

Romulan32

There is a bit of a treadmill effect here. By the time a committee agrees on the rules, the industry has already moved three steps ahead wearing a completely different pair of shoes ;)

Still, having an outdated map is better than having no map when you are exploring unfamiliar territory.

Scout

The comparison with software security is interesting. Nobody expects security standards to predict every future vulnerability, but they provide a baseline that companies have to take seriously.

AI might need something similar, where the rules evolve continuously instead of pretending one giant law will solve everything forever.

TeddyWhelan

People sometimes talk as if the EU AI Act is supposed to freeze AI development in place, but that is not really realistic. The bigger question is whether it creates useful accountability without blocking smaller players.

Large companies can usually absorb compliance costs much easier than startups, which is worth keeping in mind.

DelPiero58

The update cycle definitely feels like the weak point. Technology changes monthly while legal systems often move at a pace measured in years.

Maybe the answer is giving regulators more technical review power instead of forcing every adjustment through the entire legislative process.

Dragon49

There is a danger in chasing every new capability too quickly though. If rules are constantly rewritten based on the latest headline, you end up with unstable regulation that nobody can plan around.

Some patience is needed too, even when the technology is moving at ridiculous speed :D
sudo make me a sandwich

LegendaryLuca49

The compute threshold debate reminds me of trying to measure a car only by horsepower. Useful information, but it does not tell you how the vehicle is actually being used.

AI governance probably needs multiple measurements instead of searching for one magic number.

Shannon

A lot of people underestimate how hard it is to regulate something that is still being invented. It is not like creating rules for an established industry where the risks are already well understood.

The challenge is making rules that are specific enough to matter but broad enough to survive the next generation of models.

SockPuppet93

The EU approach has flaws, but at least it acknowledges that powerful AI systems should not operate with zero oversight. Waiting for perfect knowledge before acting has its own risks.

The hard part is avoiding a situation where only the biggest companies can afford to comply.

Context Kayla

One amusing part of this debate is that every generation of technology gets the same argument. The world changes, old rules struggle, and everyone says the new thing is impossible to regulate.

Then eventually society catches up, usually after a lot of messy trial and error.

Beta

The biggest weakness may not be the age of the rules but how quickly they can be updated. A good framework should have a built-in refresh button rather than needing a complete rebuild.

Nobody wants lawmakers trying to understand new AI systems from a five year old briefing document.
Believe.

Binary Hermit

There is also a global coordination problem here. If one region updates constantly while others do nothing, companies may just move activity elsewhere.

Technology regulation has always had this tension between local control and international competition.

QuietNomad

The "outdated immediately" argument is fair, but every standard begins outdated in some way. Building bridges and safety systems always involves making decisions with incomplete information.

The goal should be useful guidance, not a crystal ball.

Arty Leah

Some of the criticism feels like expecting a weather forecast to predict every storm decades from now. It does not need to be perfect to be valuable.

The important part is having mechanisms that notice when conditions change and respond before problems become permanent.
All original content unless stated

WaveFunction30

A lot depends on enforcement too. A beautifully written law that nobody can practically apply is just paperwork with better formatting :)

The real test will be whether companies change behavior because of it.

TinyCompass

The rapid increase in AI capability does make traditional regulation look awkward. Governments are built around careful review, while tech companies are built around rapid iteration.

Finding a middle ground is probably the hardest part of this entire discussion.
The truth is usually more complicated than the headline

Rachel

There is a reasonable argument that regulation should focus less on model size and more on consequences. A smaller system causing real harm still deserves attention.

Metrics are useful, but they should support judgment rather than replace it.

Brooke_19

Maybe the answer is treating AI rules more like medical guidelines, where experts update recommendations as new evidence appears.

That would require trust between scientists, policymakers, and companies, which is probably the hardest resource to find.

Inference Python

The pace mismatch is definitely frustrating. Watching legislation move slowly while AI announcements arrive every week feels like trying to chase a train on foot.

But abandoning regulation because it cannot move instantly would probably create even bigger problems.

Reacher Mitchell

There is a balance between being too cautious and too confident. Some people assume every AI concern is exaggerated, while others assume every new capability is automatically dangerous.

Reality is probably somewhere in the uncomfortable middle.

Stuart78

The funniest part is that everyone agrees AI changes quickly, but they disagree completely on what that means for policy. Some say move faster, others say slow down and understand more.

Both sides have a point, which is why this debate keeps coming back.

Sentinel38

There is a fair point here, but I think "outdated the day it takes effect" depends on what we expect regulation to do. Laws are almost always behind technology because they are trying to create stable rules around something that keeps moving.

The bigger problem is whether the rules are flexible enough to adapt. A framework that gets updated every few years might work better than pretending one document can predict every AI development.

Baz

The comparison with cybersecurity is interesting. Nobody says security rules failed because new threats appeared after they were written. The goal is usually to create a baseline of responsible behaviour.

AI is moving faster though, so regulators probably need a faster feedback loop. Waiting five years between major updates would be a problem.
Making the internet slightly better one post at a time

Storm52

A lot of people underestimate how difficult this is. A regulator has to write something broad enough to cover future systems but specific enough that companies know what they are expected to do.

Too vague and nobody knows what compliance means. Too detailed and it becomes obsolete almost immediately. That balance is the hard part. :)
git commit -m "fixed everything"

Undertaker_EU

There is a danger of focusing only on the speed of AI improvement and ignoring the speed of institutions. Governments were never designed to release updates like software companies do. ;)

Maybe the answer is not constant rewrites but clearer principles. Things like transparency, accountability and safety testing are likely to remain relevant even when the models change.

Taz18

A moving target does make this difficult, but waiting until everything is understood is impossible because by then the technology will already be everywhere.

A good approach might be building regulations around outcomes rather than specific tools. A rule about harmful decisions made by AI could survive longer than a rule about one particular model type.
Powerbombed my keyboard, it deserved it

MondayMoan67

There is a reasonable argument that regulation should be slower than development. We do not want every new AI feature causing governments to rush out another law before anyone understands the consequences.

Sometimes watching how technology is actually used gives better information than trying to predict every possible future scenario.

CMPunk_Mike

The global coordination issue might be the biggest challenge. If Europe creates strict rules, the US takes another approach, and other regions do something completely different, companies will have a confusing patchwork to deal with.

That said, moving everything to the least regulated location is not automatically a solution. Customers and businesses still care about trust and reliability.

Related Topics (2)

Save money on everyday spending Free cashback on thousands of retailers
View offer