Tool-using language agents increasingly modify external systems, yet most runtime safeguards allocate scrutiny only by generic uncertainty scores or coarse mutation heuristics. We introduce CASEL (Consequence-Aware Selective Execution Layer), a model‑agnostic runtime allocation policy. It combines calibrated failure probability, state‑dependent consequence, reversibility, ambiguity, and evidence freshness to choose, for each proposed tool call, among direct execution, verification, sandboxing, clarification, approval, and abstention, minimizing the cost of each mode.
We formalize direct execution as selective prediction and state the validity conditions of a finite‑sample threshold and a trajectory risk ledger. In a controlled simulation with 20 seeds and 120,000 test actions per seed, all methods share the same intervention menu and compute cost. CASEL reduces harmful‑side‑effect trajectories from 14.75% for an uncertainty‑ranked gate to 7.02% and mean normalized harm from 0.184 to 0.123. A gate ranked by the product of failure probability and consequence achieves similar harm (5.82%, 0.120) but routes 45% more actions to human approval; removing reversibility‑aware routing raises the rate to 15.53%. In contrast, a trajectory‑level risk budget increases harm to 11.82% at equal cost. Intervention effectiveness is assumed rather than measured: the results rank allocation rules under the simulator’s assumptions and do not establish deployment effectiveness.
Review