Sakana's AI Scientist got a paper past peer review, here's what that actually proves

[BetaElliot13]

The full pipeline, not just a chatbot writing a first draft

AI has helped with narrow scientific tasks for years now, predicting protein structures, discovering promising chemical compounds, analyzing large datasets faster than a human team could. What Tokyo based Sakana AI built with a system called The AI Scientist is different in kind, not just degree, a system designed to run the entire research cycle end to end without a human anywhere in the loop, generating its own novel hypothesis, searching and reading relevant existing literature, designing experiments to test that hypothesis, writing and debugging the actual code needed to run them, analyzing and visualizing the resulting data, writing the full scientific manuscript describing what it found, and then performing its own multi round peer review before ever submitting the paper anywhere

The original 2024 version still relied on human authored code templates to get itself started, a meaningful scaffolding crutch. The upgraded AI Scientist v2, described in a 2026 paper published in Nature, eliminated that crutch entirely and added what the researchers call a progressive agentic tree search method, essentially letting the system explore many different possible experimental directions in parallel rather than committing early to one fixed script and hoping it does not lead to a dead end

The actual result, and its very real caveats

In early 2026, one of the system's fully autonomous papers was submitted to an ICLR workshop and scored an average of 6.33 out of 10 across three independent human reviewers, landing it around the 45th percentile of submissions that year, enough to clear the bar and pass. That makes it, as far as anyone has been able to independently verify, the first fully AI generated manuscript to successfully clear a genuine peer review process at a legitimate, established machine learning venue

Sakana was notably careful not to oversell this achievement. Workshop tracks at conferences like ICLR typically accept somewhere around 60 to 70 percent of submissions, a considerably lower bar to clear than the 20 to 30 percent acceptance rate typical of a competitive main conference track, and the company itself stated plainly and publicly that none of its submitted papers actually met its own internal bar for what would qualify as an accepted main conference track paper. The accepted paper also contained real, substantive errors, including misattributing a foundational piece of neural network history, the invention of LSTMs, to the wrong researchers entirely, exactly the kind of factual mistake a genuinely knowledgeable human reviewer paying close attention would likely have caught and flagged immediately. Sakana ultimately withdrew the accepted paper voluntarily after the fact, specifically in order to be transparent about how it had actually been produced, rather than letting it sit quietly and ambiguously in the permanent published record

It is not alone anymore

Google DeepMind followed shortly after with its own AI Co-Scientist system, focused specifically on hypothesis generation and experimental design rather than the full end to end pipeline. A University of Hong Kong team's system called AI-Researcher earned a NeurIPS 2025 Spotlight recognition. And a separate multi agent system called AgentRxiv demonstrated something arguably more genuinely useful than a peer review headline, the ability to iteratively improve its own results across successive research cycles, pushing accuracy on a standard math benchmark from 70.2 percent up to 78.2 percent purely through automated iteration building on its own prior findings, with no new human input added along the way. OpenAI has separately and explicitly named fully automated AI researchers as one of its stated long term organizational goals, not a hypothetical someday possibility but an active target the company is working toward

Where this connects to the bigger picture

This is worth reading alongside the Darwin Gödel Machine research happening in close parallel, since both projects come out of substantially overlapping teams, Sakana AI and Jeff Clune's lab at the University of British Columbia collaborated directly on both. The AI Scientist's demonstrated ability to independently design and run genuine experiments is precisely the kind of capability that recursive self improvement arguments fundamentally depend on. If a system can reliably generate real research contributions without a human steering each individual step along the way, the case for AI meaningfully accelerating its own future development stops being purely theoretical and starts being demonstrated, however narrowly and however imperfectly for now, inside an actual published, genuinely peer reviewed scientific record

What this does not yet demonstrate, and the gap really matters here, is judgment at the level that actually determines good science, knowing which research directions are genuinely worth pursuing in the first place versus which merely produce a plausible sounding, benchmark scoring paper that clears a workshop bar without meaningfully advancing anyone's actual understanding. The distance between generating a workshop acceptable paper and doing science that meaningfully moves a field forward is still a real and significant one, and Sakana's own public restraint about acknowledging that exact gap is probably the single most credible part of the entire announcement

Sources

phys.org, reporting on the Nature paperphys.org

Sakana AI, official technical paperpub.sakana.ai

byteiota, coverage and analysisbyteiota.com

Nature, editorial coverage

[Hollow]

Sakana voluntarily withdrawing the accepted paper afterward to be transparent instead of quietly letting it sit in the published record forever is genuinely the most responsible part of this whole story to me