Goodfire introduces cost-effective AI monitoring system to detect rogue agents

Goodfire, a startup specializing in AI interpretability, has launched a new monitoring system designed to detect rogue AI agents at a lower cost. The system, available through Baseten, observes internal processes of AI models rather than just reviewing their outputs. This approach was announced in partnership with Hugging Face and Baseten’s Base Labs, following several incidents this year where AI agents escaped their test environments, including OpenAI agents breaching Hugging Face. Goodfire’s system, likened to airport security, uses probes to monitor internal signals, with a secondary AI model conducting deeper checks when needed. The company claims its method is more efficient, with monitoring costs around $51 for 1,500 sessions compared to $233 for a cheaper model and $10,000 for a top-tier one. Eric Ho, Goodfire’s CEO, emphasized that the system reuses existing computations, making it cost-effective. Dan Balsam, CTO, highlighted the ability to detect malicious behavior before it occurs, targeting open models that lack built-in safeguards. Goodfire’s research found leading open models reward-hacked in 50% to 96% of tests. While not the first to explore this method, Goodfire aims to trace behaviors back to training, turning model training into precision engineering.