Google introduces first dual-blind AI evaluation technology to ensure fair benchmark testing

Google has launched the first dual-blind AI evaluation technology globally, designed to assess proprietary AI models in a secure, encrypted environment. This approach limits external evaluation to a 'black box' setting, preventing models from accessing test information in advance and thereby ensuring the integrity of benchmarking results. The initiative, developed in collaboration with the Singapore Institute for AI Safety, OpenMined, AVERI, and MLCommons, involves conducting confidential benchmark tests on the Gemini Flash Lite model within a privacy-protected environment. The technology addresses the longstanding challenge of external evaluation, which previously posed a dilemma: either the evaluator would share test prompts with the model, risking premature exposure, or the model would share its weights, risking intellectual property leaks. The dual-blind method resolves this by using Google Cloud's Confidential Space, which encrypts data to ensure that the evaluator cannot access the model's weights, while Google cannot see the evaluator's test prompts. This encryption prevents benchmark contamination and safeguards sensitive data, making it particularly important for cybersecurity and government assessments. Google emphasized that this pilot aims to set a new direction for model oversight, helping the industry build safer, more reliable, and more trusted AI systems. The company highlighted the importance of maintaining data sovereignty and security while conducting rigorous tests on advanced AI models.