xAI, OpenAI, and Anthropic just cosigned a shared standard for how independent AI evaluators should operate

[Gareth84]

The AI Evaluator Forum, formed back in December 2025, has published AEF-1, formally titled Minimum Operating Conditions for Independent Third Party AI Evaluations, a voluntary standard laying out baseline requirements for the independence, access, and transparency any third party evaluator needs to actually provide a trustworthy assessment of an AI system's capabilities or risks. xAI, OpenAI, and Anthropic have all cosigned the standard, which is a genuinely rare instance of direct alignment between three companies that spend most of their public messaging competing rather than cooperating.

The timing lines up closely with Dario Amodei's essay from earlier in the week calling for the industry to pace frontier development, and specifically with Anthropic's own commitment to give independent evaluators permanent, employee level access to its systems as the first concrete step in that broader proposal. AEF-1 effectively formalizes what that kind of access should actually look like in practice, spelling out baseline conditions evaluators can point to when demonstrating they operated under genuinely independent conditions rather than simply taking a lab's word that an evaluation was thorough and unbiased.

A public letter accompanying the standard argues that organizations across the AI sector recognize a real transparency gap around third party evaluations specifically, since evaluations vary enormously in how much genuine access, independence, and freedom from conflicts of interest the evaluator actually had, and yet all get described publicly using similar sounding language regardless of how rigorous the underlying process actually was. AEF-1 is meant to close that gap by giving evaluators a checklist they can use to demonstrate they met a genuine baseline rather than leaving the public to simply trust whatever a lab chooses to disclose about its own evaluation process.

What makes this notable beyond the standard's specific technical content is simply the fact that it happened at all, and happened this fast. Just days after Amodei's essay generated a wave of public statements from rival lab leaders, three genuinely competing companies converged on a shared document formalizing evaluator standards, suggesting the underlying coordination and infrastructure for this kind of agreement was already substantially built before the public messaging even started this week

AI Evaluator Forumaievaluatorforum.org

[Tel]

The fact that this converged into an actual signed document within days of Amodei's essay rather than dragging out over months is the detail that stands out most to me here. Genuine multi company standards usually take a lot longer to negotiate and finalize, which suggests either the underlying groundwork was already mostly done before this week's public messaging began, or these companies moved with unusual urgency once the public pressure actually landed. Either explanation says something notable about how seriously this specific proposal is actually being taken internally

[SockPuppet42]

A voluntary standard is obviously not the same thing as binding regulation, and it is worth being clear eyed about that distinction even while acknowledging this is a genuinely meaningful step compared to where things stood just a week ago. Voluntary commitments from competing companies have a real track record of eroding once the initial wave of public attention moves on to the next story.

Still better to have a concrete checklist that evaluators and outside observers can actually point to and measure compliance against than nothing formal at all

[2026]

The AI Evaluator Forum having quietly existed since December 2025 before suddenly becoming publicly relevant this week is a good reminder that a lot of the infrastructure underlying these big public announcements gets built well out of public view long before it ever makes headlines. Feels like there is probably a whole ecosystem of similar quiet standard setting groups working on adjacent AI governance issues that most people have simply never heard of yet. Wonder what other groups like this are currently operating below the general public's radar right now

[Arty Leah]

The transparency gap this standard is trying to close, where evaluations get described using similar sounding language regardless of how much genuine independence and access the evaluator actually had, is a real and underappreciated problem that extends well beyond just AI safety specifically into a lot of other industries that rely on third party auditing more broadly. Financial auditing and environmental compliance certification have dealt with versions of this exact same credibility problem for decades already. Curious whether AEF-1's specific approach borrows any concepts from how those more established auditing industries eventually solved similar trust problems

[Cheugy]

Would like to see the actual specific checklist items in AEF-1 itself rather than just the framing around why it exists, since the real substance of a standard like this lives entirely in the specific operational details, what access counts as sufficient, how conflicts of interest actually get disclosed, and so on. A standard this vague in public summary could turn out to be either genuinely rigorous or fairly toothless depending entirely on those exact specifics. Going to reserve full judgment until someone actually walks through the complete document line by line

[Keira85]

Getting xAI, OpenAI, and Anthropic to actually agree on anything jointly is notable given how openly and often these three companies have been feuding publicly over model comparisons, safety philosophy, and general industry positioning over the past year. Whatever genuine competitive tension exists between them clearly did not prevent this specific narrow technical agreement on evaluator standards from actually happening. Suggests there may be more room for this kind of narrow technical coordination even amid broader ongoing rivalry between these companies than public messaging usually implies