When a tool call times out, the agent perceives a failure but receives no structured error payload. Conventional workarounds return a cached error page or a negative price in the expected format, allowing the agent to treat it as factual data. This paper introduces Outcome Monitors, which detect violations of outcome contracts mined from task‑disjoint execution traces or derived from public schemas. Upon detecting a breach, the monitor preserves the original result and emits a non‑binding receipt that names the violated property and lists publicly available recovery tools.
In frozen, pre‑specified evaluations with injected failures, we benchmark four models (two families, two models each) on the ToolMaze suite. Outcome Monitors raise completion from 10.9% to 28.1%. On the tau‑bench retail benchmark, completion improves by 14.0 and 12.0 points on two tiers respectively.
Control experiments show that removing the recovery‑tool list eliminates the gain, while restoring it recovers the effect; diagnostic detail and timing produce no measurable differences. Gains concentrate where the fault blocks completion. Using a suite derived from a published incident taxonomy, detection outside the mined vocabulary drops to 46%, yet delivery continues and overall completion remains unchanged. Recovery tools themselves constitute the active receipt content; extending detection beyond the contract vocabulary remains an open challenge.
Blogger's Review: Outcome Monitors provide a pragmatic safety net for LLM‑driven tool usage by coupling explicit contract verification with recoverable receipts. The empirical gains across multiple models and tasks demonstrate strong potential, especially in fault‑prone toolchains. Expanding contract vocabularies could further boost detection rates and system robustness.