Explanations for failures of long‑horizon agents are usually presented as taxonomies over completed trajectories: a reviewer reads the full transcript and labels the failure. We argue that attributing an agent's behavior to its history must be backed by intervention‑verified evidence, either per instance or via a validated estimator. The procedure is to freeze an executable mid‑trajectory state, alter only the history visible to the agent, and then continue execution. Crucially, this is an executed counterfactual, not inferred from the transcript, and it must be accompanied by a replay certificate measured at the restored state and matched to the estimand.
We identify four manipulation classes, each with precedents. Existing methods such as DoVer, REFLECT, the work by Min et al., Who&When Pro, and Causal Agent Replay already perform interventions from checkpoints, re‑executed prefixes, or recorded conversation states, yet none report a measurable replay certificate.
Two cautionary measurements illustrate the gap: a carrier taxonomy whose “concentration” could not be separated from benchmark construction (adjusted odds ratio 1.15, confidence interval crossing 1), and hash‑identical snapshots restored into identically seeded processes that agreed in all 24 states with each other but diverged from the live continuation in 9 and 20 of 24 runs. Restoring the captured position of the random stream made all 24 runs agree with the live execution.
Review: The paper highlights the necessity of executed counterfactuals for causal attribution in long‑horizon agents, proposes a concrete reporting framework, and could significantly improve reproducibility and trustworthiness in this research area.