Building trustworthy model benchmarks requires that models have no exposure to test data before evaluation. Just as a student who sees exam questions in advance produces a meaningless score, an AI model that has already seen benchmark prompts suffers from contamination. To address this, Google partnered with the Singapore AI Safety Institute, OpenMined, AVERI and MLCommons to launch the first double‑blind evaluation framework. The evaluation runs inside Google Cloud’s Confidential Computing Trusted Execution Environment (Confidential Space), where the external test set and the proprietary model weights are each encrypted. The evaluator cannot view the Gemini model weights and Google cannot view the test prompts; interaction happens only inside the cryptographic “box”. This provides cryptographic proof that neither side’s data is exposed, preventing models from optimizing on the test set ahead of time, which is especially important for cybersecurity or government‑level assessments. Double‑blind evaluation removes the traditional trade‑off of handing over prompts or model weights, allowing independent parties to rigorously test frontier models without compromising data sovereignty. We hope this pilot sets a new frontier for model oversight, helping the industry build safer, more reliable and widely trusted AI systems. For methodology and results, see the technical report.
Blogger's Review: In short, double‑blind evaluation leverages cryptographic guarantees to eliminate benchmark contamination, raising the trust bar for AI assessment and offering a promising path for broader adoption.