Evaluating conversational agents that operate on live, mutable enterprise data is extremely challenging: replayed sessions drift as backend state changes, and a benchmark built today can silently become stale as the underlying data evolves.
We introduce an offline evaluation harness that remains reproducible even when the live data it depends on changes, and that can be regenerated on demand once the benchmark becomes outdated.
The harness serves as the fitness function for GEPA‑based prompt optimization, with a correctness gate that guards against reward‑hacking attacks.
Applied to a production merchant‑facing agent, correctness‑gated compression reduces the full prompt surface by 3.5%‑7.6% without regressing held‑out task correctness across three independent model families, and for one family correctness measurably improves.
These findings demonstrate that offline reproducible evaluation can provide a trustworthy safety layer for prompt compression.
Review