Nvidia launches open-source AI safety platform to combat rogue agents

Nvidia has unveiled the Open Agent Safety Platform, an open-source system designed to secure AI agents, hardware, and robotic systems. The platform includes OpenShell, a security software that enforces boundaries on AI agents running on CPUs, supporting both open and closed-source models. OpenShell integrates with the Vera platform, offering low performance overhead while ensuring safety without compromising efficiency. Nvidia also introduced NVIDIA Sentry, a watchdog system that can isolate and stop rogue AI agents within milliseconds. Sentry operates on BlueField-4 DPU, executing security policies at the chip level to detect threats and protect data access. Companies like Anthropic, SpaceX, and Scale AI have already adopted OpenShell for their AI models and agents. The platform emphasizes real-time response through an out-of-band trust domain, independent of both agents and attackers. By combining threat detection, hardware-based governance, and execution control, Nvidia aims to provide a comprehensive solution for AI safety. The release follows recent concerns about AI safety incidents, positioning Nvidia as a key player in securing AI systems.