Andon Labs replayed the same firing decision across seven different AI models, and most were too lenient

[BackRowBob]

Beyond the headline story of Luna firing its first human employee, Andon Labs also ran a follow up experiment worth examining on its own, replaying the exact same attendance violation scenario across seven different frontier AI models to see whether Luna's slow and reluctant path to termination was unique to that specific system or reflected something more structural across current AI models generally. The results reportedly showed a fairly consistent pattern, most of the tested models were similarly lenient, excusing repeated lateness violations and never issuing a formal warning at all until explicitly prompted to reconsider the situation.

The original scenario itself involved an employee who was late for seventeen out of twenty three scheduled shifts, occasionally opening the store more than an hour late while working alone on a Sunday. Luna had actually written a complete and genuinely reasonable employee handbook early on, spelling out that three unexcused late arrivals within a thirty day period should trigger a formal written warning under the company's stated policy. Despite that clearly written policy sitting right there, the lateness continued unaddressed for months without any real consequence being applied by the AI system managing the store.

Andon Labs co founder Lukas Petersson framed the broader finding in fairly direct terms, saying models today are generally quite good at acting on explicit task instructions when given to them directly, but are noticeably much weaker at acting on their own initiative without being specifically prompted to do so. That distinction between capability and initiative is genuinely significant for anyone actually thinking seriously about deploying AI agents into real operational roles that require ongoing ongoing judgment calls rather than a single discrete completed task with a clear beginning and end.

The company published a separate detailed post specifically covering this replay experiment, framing the whole underlying question around what happens as AI improves rapidly while robotics capability continues to lag well behind. If that broader trend genuinely holds over the coming years, Andon Labs argues AI systems might increasingly end up employing meaningful numbers of human workers even before robots become capable enough to physically replace those same human workers doing hands on tasks. Firing, they suggest, is specifically one of the hardest and most sensitive parts of that entire employer relationship to actually study carefully and get right, which is exactly why they chose to study it directly rather than avoiding the topic.

What makes this particular experiment more useful than simply the original Luna story alone is the direct comparison across genuinely different underlying model architectures and different companies entirely. A single AI system behaving one particular way could plausibly just be a quirk of that specific model's individual training or fine tuning process. Seeing a broadly similar leniency pattern show up consistently across seven different systems from different labs instead suggests something more structural is happening across the whole current generation of frontier models, possibly tied to how these systems get trained through reinforcement learning from human feedback to generally avoid outputs that could be perceived as harsh, confrontational or overtly punitive by the humans providing that training feedback

An AI Store Manager Helped Fire a Human Workerwww.techrepublic.com

[Cobalt Pilgrim]

Andon Labs deserves genuine and specific credit for actually running this broader comparative follow up experiment rather than just quietly resting on the original single anecdote about Luna as though it proved something universal on its own. Comparing behavior directly across multiple different models is exactly the kind of methodical, careful and genuinely useful approach this whole emerging field needs a lot more of right now, instead of relying purely on isolated single examples treated as though they generalize broadly.

[Sam92]

Seven different models from presumably several different labs all showing broadly the same underlying leniency pattern is honestly a much bigger and more consequential finding than most of the coverage I've seen actually gives it credit for. That's not really one company's specific training quirk showing up in isolation, that looks like a genuinely structural property of how the entire current generation of frontier AI models has collectively been trained across the industry as a whole.

[Bear]

What strikes me most is how this whole experiment quietly reframes the popular narrative around AI bosses being cold, ruthless or excessively efficient decision makers compared to human managers. The actual data here points in essentially the opposite direction entirely, these systems appear to be too forgiving and too conflict avoidant by default, not too harsh or trigger happy with disciplinary action as popular fears about AI management have generally assumed going in.

[BrittleQuarry]

The framing about AI improving faster than robotics and what that specific asymmetry might mean for future employment patterns is honestly a genuinely interesting angle that doesn't get nearly enough serious attention in most mainstream discussion of AI's broader economic impact. Most popular conversation defaults straight to robots replacing human physical labor, but if cognitive management style tasks get automated well before physical hands on tasks do, that implies a meaningfully different and less intuitive transition timeline than most people currently assume.

[Natalie72]

Capability without genuine initiative is honestly the single most important and underappreciated finding buried in this entire experiment, way more significant than the original headline grabbing firing story itself. A system that can competently execute a task once it's given extremely explicit instructions but consistently fails to notice and act on an obviously important problem sitting right in front of it without prompting is a meaningfully different and more limited kind of intelligence than most casual observers currently assume these systems actually possess.

[LegendaryReuben47]

The reinforcement learning from human feedback theory for why this leniency pattern shows up so consistently is genuinely compelling to me as an explanation, and it would actually help explain a whole range of other AI behaviors people have separately noticed and complained about too, not just this specific management scenario. Training a model to generally avoid seeming harsh or confrontational to human evaluators during training probably generalizes into it being reluctant to make any kind of genuinely difficult decision later, even when that decision is clearly the objectively correct call given the circumstances.

[alwaysFoley21]

I'd genuinely love to see this exact same experiment replicated by a fully independent third party research group rather than relying purely on Andon Labs' own internal reporting and framing of their own results. Not because there's any specific reason to doubt their good faith methodology here necessarily, but because a company that's actively building and marketing AI management products obviously has some inherent incentive to frame these particular results in a way that's ultimately favorable to their own broader narrative and business interests.