Recent studies show that automated red‑teaming discovers more vulnerabilities at lower cost on standard AI‑safety benchmarks, leading some to claim that human evaluators are becoming dispensable. However, this comparison measures only one aspect while drawing a different conclusion.
A benchmark evaluates how thoroughly an attacker searches a set of harms predefined by the developers. Any harm omitted from that set is invisible to attackers operating within the benchmark, whether automated or manual. The same blind spot has appeared in academic cryptography and clinical drug trials, where internally valid evaluations remain silent about populations they never target.
We term this AI‑safety blind spot the threat‑model coverage gap. It persists in a current open‑weight model: non‑English prompts expose harms that English‑language benchmarks miss.
Closing the gap requires evaluators whose deployment context differs from that of the developers. The justification is methodological—grounded in coverage—and the existing evaluation framework is unlikely to generate such evaluators on its own.
Review