Agent platforms are beginning to improve their own agents: they read execution traces of a deployed agent, derive a change, have an automated verifier approve it, and release it without human involvement. We ask how much of this loop can run safely unattended. Two measurable quantities are introduced: verifier strength $\\sigma$, which measures how well the available verifier (often an LLM judge) agrees with a trusted reference verifier and limits how much a step can improve the agent; recoverability, which gauges how cheaply a bad change can be undone, including whether a new verifier approval is needed and how much of the old behavior a rollback restores. A step should run unattended only when both are sufficient. We implemented this rule on our platform, allowing prompt, tool, and skill edits to run unattended while weight updates still require human sign‑off. The main contribution is the definitions and measurement methods for these two quantities. We built a platform that enforces the rule and evaluated it in two ways: on the platform’s own approval gate and in controlled experiments on the $\\tau^2$ benchmark. Regarding verifier strength, the platform’s LLM judge gave the same score across 9 of 37 settings regardless of input and reported no error, showing that a verifier can appear healthy while approving anything; two simple tests catch such failures. Concerning recoverability, restoring a saved copy returned the agent to its original behavior, whereas a regenerated undo did not; during a deployment the change altered the outcome of 17% of tasks beyond normal run‑to‑run variation, and no undo repaired it. Together these findings support the rule: a change may run unattended only when its old version is kept and the verifier’s strength has been measured.
Review