Ant group unveils open-source tool for real-time risk detection in large models

Ant Group's AI Safety Lab has open-sourced SingProbe, a real-time risk detection tool for large models. It uses hidden states from the base model's reasoning process to assess risks at the token level with less than 0.5% additional computing overhead. SingProbe can perform intent classification, safety detection, and hallucination identification simultaneously. It is compatible with 29 mainstream large models and is the first complete intrinsic safety technology solution open-sourced by a major tech company. The open-source code and pre-trained models were released together, showing significant advantages in high-risk scenarios such as healthcare, enabling millisecond-level blocking before content output. The release of SingProbe marks a significant step in AI safety, offering a lightweight and efficient solution for real-time risk assessment. By leveraging the base model's hidden states, the tool minimizes computational overhead while maintaining high accuracy. This approach differs from mainstream external safety review models, which typically require separate systems for risk assessment. Ant Group's open-source initiative aims to enhance transparency and collaboration in the development of safe and reliable large models. SingProbe's open-sourcing is expected to influence the broader AI community by providing a practical framework for integrating intrinsic safety measures into large model development. This move aligns with growing industry efforts to address risks associated with advanced AI systems, particularly in sensitive domains like healthcare. The tool's ability to enable millisecond-level blocking highlights its potential to prevent harmful outputs in real-time, contributing to safer and more responsible AI deployment.