Large language model (LLM) agents tune GPU kernels and serving engines via a propose‑measure‑keep loop, yet the measurements behind this loop are often untrustworthy. From a four‑day pilot corpus of 619 model calls we identify four failure modes: strawman baselines fabricate speedups, absolute times do not transfer across machines, saturated tasks nullify comparisons, and infrastructure defects masquerade as scientific results.\ \ AutoTuneBench addresses these issues with a benchmark and measurement protocol that makes trust an architectural property. The protocol is frozen as code with test‑enforced provenance; a database‑level validator rejects out‑of‑protocol results; anti‑cheat checks run outside the agent's modification surface; comparisons follow pre‑registered readouts; measurements anchor to externally published results and rely on paired‑seed statistics with a cross‑run coefficient‑of‑variation cap of $5\%$.\ \ Under this framework the headline numbers are substantially revised: the best kernel reads $10.6\times$ against a naive baseline but $2.03\times$ against the honest one; one configuration yields $1.174\times$ on machine A and $1.0049\times$ on machine B; a pre‑registered on/off comparison nulls at a shared wall of $2.4840\,$ms versus $2.4957\,$ms; the KernelBench Level‑1 suite admits $51\%$ of tasks with a median speedup of $1.0001\times$ over PyTorch eager.\ \ The protocol, the two‑engine corpus (vLLM and SGLang), and the full audit trail are released as open artifacts.\ \ Review: AutoTuneBench provides a rigorously defined, reproducible measurement pipeline that eliminates cheating in automated tuning, establishing a trustworthy baseline for LLM serving performance and setting a new standard for future research.