LLM agent fleets can now perform actions that cannot be fully undone, such as moving money, deploying code, deleting data, and disclosing information. Existing controls typically check a single effect at a time, which means that even if every local gate remains correct, a fleet of individually authorized agents can still exceed a principal's risk limit under a shared trigger. We introduce the irreversibility budget, a cumulative residual value‑at‑risk account maintained by a trusted runtime for each principal across agents, workflows, and tenants.
In this model, irreversibility is treated as a first‑class resource. The runtime charges each effect its residual loss at the agent level and denies the marginal effect once the aggregate cost would overdraw the budget.
Pricing is challenging because effects are heterogeneous, may be adversarially declared, and can be correlated. In a controlled study we found that, while per‑effect gates stay correct, fleet‑level overdraws can reach up to 48 times the tenant's risk limit, yet the budget keeps every correctly charged run within that limit.
Nevertheless, devising a conservative, dependency‑aware pricing scheme remains the central open problem for a deployable design.
Review