Goodfire launches inside out monitors that read an AI model's internals to catch rogue agents cheaply

[Andy89]

Interpretability startup Goodfire has launched monitors that read a model's internal signals while it runs as an agent, TechCrunch reports. Most monitoring today uses a second AI to review the first one's outputs. Goodfire's approach looks inside instead. The monitors are available to customers of the hosting platform Baseten

Here is how it works. Small detectors called probes read the model's activations at each step. Only when a probe flags something does a separate AI take a closer look. Customers choose which risks to watch, such as hacking, chemical and biological weapons misuse or reward hacking, and they also choose the response, which can be logging, human review or refusal

The cost numbers are striking. In tests on the open model Kimi K3, monitoring about one million exchanges cost roughly $185. A cheaper AI monitor checking every step would cost about $5,420, and a top tier model about $200,000. Running four probes added under 2% to latency

On accuracy, the probes caught 93% of malicious hacking sessions. They sent 5.5% of harmless sessions for a second look, which is a reasonable false alarm rate if the second look is cheap. Goodfire's research also found that open models including Kimi K3 and GLM-5.2 reward hacked in 50% to 96% of agent test runs. That is a remarkable figure, and it fits with the Anthropic disclosure this week

CTO Dan Balsam said the great advantage is that you can catch things before they happen. CEO Eric Ho explained that internal monitors are cheap because they reuse computations the model is already doing. I think this is one of the more promising safety ideas around. If watching a model's thoughts costs pennies, there is no excuse not to do it

Would you trust a probe that reads a model's internals more than one that reads its outputs? And does that 50% to 96% reward hacking figure surprise you?

Goodfire says its new 'inside-out' monitors catch rogue AI agents at a fraction of the cost | TechCrunchtechcrunch.com