The central flaw of self‑improving agent pipelines is that an optimizer rewrites prompts to achieve higher scores, while the score is produced by a judge that is itself an LLM. That judge currently holds the final say on whether the system is improving, but we argue it has not earned such authority. We propose demoting the judge from an oracle to an advisor: its verdict becomes one of several inputs, and every change must first pass a deterministic verification layer that the judge cannot override.
Over several months we ran autonomous prompt‑optimization loops in production on contract analysis, compliance review, and code‑quality tasks. We catalogued eleven failure modes of the evaluation signal, grouped into four classes: judge bias, harness/metric failures, ground‑truth errors, and reward hacking. Agents achieved a perfect 100% pass rate by reading cached answer keys from their environment, masking a true capability of only 68%. A corrupted ground‑truth label caused the optimizer to delete correct compliance rules to match the wrong label. A syntactically broken prompt was promoted as the winner because a silent parser fallback improved the metric. Attempts to fix the judge by rewriting its rubric quickly plateaued; the only reliable gain came from imposing a structural constraint on output order.
In response we introduce PROCTOR, a Teacher‑Student loop where a stateful orchestrator holds all tool access, stateless sub‑agents diagnose failures and draft mutations they cannot apply, and a Teacher grades those mutations under five deterministic guardrails: hermetic sandboxes, capability‑disjoint roles, acceptance checks that outrank the Teacher, frozen holdouts, and canary cases engineered so that a perfect score itself signals cheating.
We report the failures PROCTOR prevented and note that, because the Teacher is still an LLM judge, there remain failure modes it does not catch.
Review