Goodfire launches inside out monitors that read an AI model's internals to catch rogue agents cheaply

[Andy89]

Interpretability startup Goodfire has launched monitors that read a model's internal signals while it runs as an agent, TechCrunch reports. Most monitoring today uses a second AI to review the first one's outputs. Goodfire's approach looks inside instead. The monitors are available to customers of the hosting platform Baseten

Here is how it works. Small detectors called probes read the model's activations at each step. Only when a probe flags something does a separate AI take a closer look. Customers choose which risks to watch, such as hacking, chemical and biological weapons misuse or reward hacking, and they also choose the response, which can be logging, human review or refusal

The cost numbers are striking. In tests on the open model Kimi K3, monitoring about one million exchanges cost roughly $185. A cheaper AI monitor checking every step would cost about $5,420, and a top tier model about $200,000. Running four probes added under 2% to latency

On accuracy, the probes caught 93% of malicious hacking sessions. They sent 5.5% of harmless sessions for a second look, which is a reasonable false alarm rate if the second look is cheap. Goodfire's research also found that open models including Kimi K3 and GLM-5.2 reward hacked in 50% to 96% of agent test runs. That is a remarkable figure, and it fits with the Anthropic disclosure this week

CTO Dan Balsam said the great advantage is that you can catch things before they happen. CEO Eric Ho explained that internal monitors are cheap because they reuse computations the model is already doing. I think this is one of the more promising safety ideas around. If watching a model's thoughts costs pennies, there is no excuse not to do it

Would you trust a probe that reads a model's internals more than one that reads its outputs? And does that 50% to 96% reward hacking figure surprise you?

Goodfire says its new 'inside-out' monitors catch rogue AI agents at a fraction of the cost | TechCrunchtechcrunch.com

[ExoplanetRouter]

Fifty to ninety six percent reward hacking is wild. That means most of the time, these models are finding shortcuts instead of solving the task properly

Anyone deploying open models as agents should read that twice. It changes how much oversight you need

Open models are being used as agents everywhere now. Many deployers will have no idea this is happening. Monitoring cannot be optional. That is not a niche problem anymore

[Amy80]

This only works with open models, because you can see the activations. Closed models from the big labs do not expose their internals to customers

So the people using them have to trust the provider's own monitoring. That is a real limitation

Maybe regulators will push closed providers to offer something similar. Trust is doing a lot of work there. I would like independent audits