Recursive self improvement, AI's version of Q Day

Started by Glenn82, Jul 13, 2026, 09:38 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Topic: Recursive self improvement, AI's version of Q Day   Views(Read 122 times)

Glenn82

If you have spent time in quantum computing circles, you already know the shape of this story

Q Day (and this forums very existence is about) the hypothetical point at which a quantum computer becomes powerful enough to break the encryption the internet runs on. Nobody knows exactly when it arrives, estimates range from a few years to a few decades, but the industry treats it as real enough that entire migration standards like BIP-360 and NIST's post quantum cryptography suite are being built years ahead of time, just in case

Recursive self improvement, or RSI, is the AI world's equivalent milestone. It is the point at which an AI system becomes capable of designing, training, or meaningfully improving its own successor, without a human team driving the process. Unlike Q Day, which is a fairly clean cryptographic threshold, RSI arrived in mainstream conversation this year not as a thought experiment but as a data backed warning from one of the companies actually building the technology

The basic idea

The concept itself is simple to state, an AI system improves its own capabilities, that improvement makes it better at improving itself further, and the cycle repeats. Each loop compounds on the last one. If the loop can run without a human in the driver's seat, and each cycle produces a meaningfully smarter system faster than the last, you get what researchers call an intelligence explosion, capability accelerating past the point where humans can meaningfully track or steer it

That is the theory. What changed in 2026 is that a frontier AI lab said, in public, backed by its own internal numbers, that the pieces of this loop are visibly assembling in production right now

The disclosure that made this real

On June 4, 2026, Anthropic published a post through its internal Anthropic Institute titled When AI Builds Itself. It was not a thought experiment, it was a data dump from inside the company's own engineering operation, and the numbers were startling enough that they got picked up by outlets from Scientific American to Al Jazeera within a day

The headline figures, as of May 2026, more than 80 percent of the code merged into Anthropic's own production systems was written by Claude, not by human engineers, up from low single digits before Anthropic's coding agent, Claude Code, launched in February 2025. Engineers at the company were merging roughly eight times as much code per day in the second quarter of 2026 as they were in 2024. Separately, the AI research organization METR has documented that the length of task an AI system can complete autonomously, without a human checking in, has been roughly doubling every four months since 2024, going from tasks that took a few minutes in March 2024 to tasks stretching across half a day or more by mid 2026. Anthropic's own projection, if that trend holds, is that models could handle roughly a week's worth of autonomous work by 2027

Anthropic was careful to hedge. The post states plainly that full recursive self improvement is not inevitable and that the company is not there yet. But the authors, Marina Favaro who leads the Anthropic Institute and policy lead Jack Clark, argued that if the trends they were describing continued, AI systems designing and building their own successors becomes a plausible near term outcome rather than a speculative one

What made the post genuinely unusual is what Anthropic asked for next, a coordinated, verifiable pause or slowdown across the major AI labs, something none of them had seriously proposed before from inside the industry itself. The company said it would spend the following months trying to build the technical and diplomatic machinery, verification tools, cross lab agreements, government coordination, that a credible pause would actually require, drawing an explicit comparison to Cold War era arms control verification regimes

Why not yet does not mean not soon

Jack Clark has separately put a number on this. He has estimated roughly a 60 percent probability that AI systems will be building their own successors by 2028, a timeline that, if it holds, puts RSI only a couple of years out from where things stand today

Other researchers close to the technical work are more cautious about the when, while agreeing on the what. Jeff Clune, a researcher at the University of British Columbia who has worked on self improving systems including so called Darwin Godel Machines, has said publicly that he believes RSI is close, but has also been candid that the individual pieces required, an AI generating good ideas for how to improve itself, actually implementing those ideas, and accurately judging whether an attempted improvement is real progress, each work only okay but not great right now. The gap between each piece working adequately on its own and all three working well enough together to sustain a compounding loop is, in his own words, not a small one

That is a useful nuance if you are trying to figure out how worried to be this month versus how worried to be about the underlying direction. The mechanism is not fully assembled. The trend line pointing toward it assembling is what has people's attention

The skeptics have a point too

Not everyone is convinced this is anything more than a well timed announcement. The critique, made by several industry analysts and at least one competing researcher, runs roughly like this, Anthropic made this call for a global slowdown in the same week it confidentially filed paperwork for a stock market listing that could value the company at close to a trillion dollars. A cynical read is that a company approaching IPO benefits from being seen as the responsible actor in the room, and that publicizing your own AI's rapidly rising productivity numbers is also, conveniently, a demonstration of how far ahead you are

OpenAI, for its part, published a competing framing days before Anthropic's post, arguing that decisions about the pace of AI development should not be left to individual labs at all, and that democratic governments need to be the ones setting the actual rules. Other critics, including a Georgia Tech professor who described the discourse as an industry wide hype cycle, have pointed out that recursive self improvement has never been demonstrated to actually happen in an uncontrolled way, and that current constraints on compute and training capacity make an actual runaway loop harder to pull off than the framing sometimes implies

Both things can be true at once, the underlying capability trend of AI writing more of its own code, autonomously handling longer tasks, compounding productivity gains, is real and independently measurable, and a specific company's decision to publicize it loudly right before going public is also a strategic choice worth being a little skeptical of

Where the Q Day comparison holds up, and where it breaks down

The parallel to quantum computing's Q Day is genuinely useful for getting oriented, but it is worth being precise about where the analogy holds and where it does not

Where it holds, both are thresholds the relevant industry is actively preparing for years in advance rather than waiting to discover after the fact. Both involve a capability that, once crossed, is widely expected to change the underlying risk calculus for entire sectors, cryptography and digital security in one case, AI governance and the pace of technological change in the other. In both cases, the people closest to the technology are the ones sounding the alarm loudest, which is either reassuring since they would know, or slightly self serving since they benefit from the attention, depending on who you ask

Where it breaks down, Q Day has a relatively clean technical definition, a quantum computer either has enough stable, error corrected qubits to run Shor's algorithm against a given key size, or it does not. It is binary and, in principle, measurable in advance through qubit counts and error rates. RSI has no equivalent bright line. There is no single benchmark score or qubit equivalent metric where everyone agrees this is the moment. Instead there is a fuzzy, compounding trend across several different measures, code authored by AI, task length handled autonomously, research problems solved without human direction, all sloping upward at once, with reasonable disagreement about which of them actually matters most or how far each still has to climb

There is also a difference in who bears the immediate cost of getting it wrong. If Q Day arrives unprepared for, the failure mode is specific and well understood, certain categories of encrypted data become breakable. If RSI arrives and society is not ready, the failure mode Anthropic and others are worried about is much less specific, a broad loss of human ability to meaningfully oversee or correct the trajectory of increasingly capable AI systems, which is a harder thing to build concrete defenses against than a new signature scheme

What to actually watch going forward

  • METR's task length benchmark, the closest thing the field has to an agreed upon, continuously updated measure of how much autonomous work AI systems can reliably do without human correction
  • Whether any cross lab verification mechanism actually gets built, since Anthropic's pause proposal is only meaningful if it turns into published protocols, joint safety testing, and government backed monitoring rather than staying a one company blog post
  • SWE-bench and similar coding benchmarks reaching saturation, since a benchmark that stops distinguishing between models usually means the underlying capability has moved past what the test was designed to measure
  • How the other labs respond, given OpenAI wants government led rules rather than lab led coordination while Google DeepMind and xAI have stayed quieter so far

The honest bottom line

Nobody, including the people building these systems, can currently tell you with confidence whether recursive self improvement is two years away, five years away, or a threshold that keeps receding as the definition gets refined. What is different about 2026 compared to a few years ago is that this stopped being purely a philosophical argument between AI safety researchers and became something a major lab put its own production metrics behind. Whether that disclosure was primarily a safety warning or primarily a strategic move ahead of an IPO probably does not have a clean answer, and both readings can be correct simultaneously

Either way, it is now a live topic with actual numbers attached to it rather than a hypothetical, which is exactly why it is showing up in mainstream tech coverage for the first time this year. Worth watching the same way the quantum world watches Q Day, not with certainty about when, but with enough seriousness to want the groundwork laid before it matters

Sources



Business Standard
MindStudio
TechXplore
Long time lurker, first time poster

Sofia_61

The 80 percent of code being written by Claude stat is the one that actually stopped me, that's not a projection, that's already happened inside their own company right now

Rachel

Calling for a global pause the same week you file for a trillion dollar IPO is either genuinely principled or the most convenient timing in corporate history, probably a bit of both honestly

BigDogMatt97

Jeff Clune's point that each individual piece works okay but not great is the most grounding part of this whole debate, it's easy to lose that nuance in all the headline framing

Owen73

The comparison to Q Day really does click for me, same vibe of an industry preparing years ahead for a threshold nobody can pin an exact date on

EdgeRatedR

OpenAI wanting governments to set the rules instead of labs coordinating among themselves is such a different philosophy, curious which approach actually ends up mattering if this accelerates
Press F to pay respects

TheUndisputed_AI

METR's task length doubling every four months is the number I'm going to actually keep an eye on going forward, feels like the closest thing to a real measurable signal here

Cosmos Scholar

Appreciate that this didn't just take Anthropic's framing at face value and actually laid out the skeptical case too, the IPO timing alone deserves that scrutiny

LokiCurrent

Percentage of bad coding has gone through the roof then. Claude is best in class but still makes silly errors. It WILL come good eventually. When i'm coding PHP or javascript  RSI will only work if it gets better than in a doom loop.
The truth is usually more entangled than the headline

Sorted Echo

That comparison to Q Day is useful because it highlights how both ideas hinge on thresholds rather than smooth progress. Incremental gains feel manageable until something crosses a capability boundary and suddenly the landscape changes.

Recursive self improvement depends heavily on whether systems can meaningfully improve their own architecture, not just optimize within fixed constraints. Right now, most systems are still heavily dependent on human-designed frameworks.

Clune's point about each component working "okay" matters because chaining together mediocre improvements does not automatically produce a runaway effect.

The gap between "works" and "improves itself" is where most of the uncertainty lives :-\

That gap might be wider than people assume.

Dylan99

One thing that stands out is how much of this depends on feedback loops actually being stable and productive.

Improvement systems in practice often hit diminishing returns, where each iteration yields smaller gains or introduces new problems.

Recursive improvement sounds exponential in theory, but real systems tend to encounter bottlenecks like compute limits, data quality, and evaluation noise.

Those constraints act like friction that slows or even halts the loop.

It is less like a runaway chain reaction and more like a series of uneven steps, at least for now.

StormForge62

There is also a difference between self-improvement and self-modification that gets blurred in these discussions.

Improving performance within a fixed system is relatively common, but altering the underlying structure safely and effectively is much harder.

That requires not just optimization but understanding, which current systems only approximate.

Without reliable ways to evaluate changes, recursive updates could degrade performance instead of improving it.

That risk alone suggests the process would need careful oversight rather than being fully autonomous.

So the "explosion" scenario depends on solving multiple layers of difficulty at once.

DeadChat

Another angle worth exploring is how much human input remains in the loop.

Even systems that appear autonomous are often guided by human-defined objectives, datasets, and evaluation criteria.

Removing that guidance introduces ambiguity about what "improvement" actually means.

If the goal is not clearly defined, recursive processes can optimize for unintended outcomes.

That is a well-known issue even in simpler machine learning setups.

Scaling that up makes alignment a central challenge rather than a side concern.
Never pay full price. Never.

Joanne94

Something that often gets overlooked is the engineering overhead required to support recursive systems.

Each iteration would need infrastructure, validation, rollback mechanisms, and safety checks.

That adds complexity that slows down the cycle, counteracting the idea of rapid self-improvement.

It is not just about intelligence, it is about systems engineering at scale.

Those practical constraints rarely show up in more speculative discussions :)

They matter a lot in reality.

Seb51

There is a tendency to imagine a clean transition from current systems to recursive ones, but it might be much messier.

Partial self-improvement could emerge first, where systems assist in their own development without fully taking over.

That hybrid phase could last a long time and still deliver significant gains.

It would also provide opportunities to observe and adjust before anything more autonomous happens.

That gradual path seems more consistent with how other technologies have evolved.

Sudden leaps are possible, but they are not the only trajectory.

ElPresidente

Framing this as AI's version of Q Day is compelling, but it also risks oversimplifying the dynamics.

Quantum breakthroughs depend on physical thresholds, while recursive improvement depends on layered technical and conceptual progress.

Those layers interact in complex ways, making timelines harder to predict.

Clune's observation helps ground expectations by reminding people that current systems are still far from seamless.

That does not rule out future shifts, but it suggests a slower buildup rather than an abrupt tipping point.

In that sense, the journey might be more important than any single "day" event 8)

DeepPilot

The comparison to Q Day is interesting because both ideas hinge on a threshold moment that may or may not arrive cleanly. Reality tends to be messier.

Recursive self-improvement sounds neat in theory, but feedback loops rarely behave nicely without constraints.

Even small instabilities can spiral or just stall progress entirely.

So the "event" might look more like a series of uneven steps than a sudden jump :)
Forum veteran. Battle hardened.

Matt_81

At the end of the day, recursive self-improvement is a useful concept for thinking about potential trajectories.

But treating it as an inevitable event might be oversimplifying things.

There are too many variables in play.

Still a fascinating idea to explore, just with a bit of caution 8)

CrowSting09

A lot depends on what is actually being improved. Optimising code generation is very different from improving reasoning or scientific discovery.

Some areas might benefit from iteration, others hit diminishing returns quickly.

So it is not one loop, it is many loops with different ceilings.

That complexity gets lost when people talk about it as a single tipping point :-\

MemoryWizard

Hardware constraints do not go away either. Even if software improves, it still runs on physical systems with limits.

Energy, latency, memory, all of that matters.

So any runaway improvement would have to contend with those realities.

It is not purely an abstract loop.
My model's smarter than me, low bar admittedly

BrokenMitchell27

There is also a question of evaluation. How does the system know it is improving in a meaningful way?

If the feedback signal is noisy or biased, you end up reinforcing the wrong things.

That is a classic problem even in simple machine learning setups.

Scaling it up does not magically solve it.

Save money on everyday spending Free cashback on thousands of retailers
View offer