The deployment of AI agents in long‑horizon tasks generates massive execution logs. Ensuring reliability requires diagnosing failures within these logs, turning outcome‑level signals into actionable interventions. Because the data volume makes manual review infeasible, automated root‑cause attribution (RCA) becomes essential. Existing RCA approaches that rely on large language models (LLMs) suffer from low diagnostic accuracy, especially as execution traces grow longer. Relevant information is often sparse, scattered across distant actions, and disconnected from the visible failure, turning RCA into a massive search problem. Traditional methods typically use a one‑shot LLM judgment to diagnose from the full trace. While this works for short trajectories, the judge often converges on a plausible diagnosis too early, leaving critical evidence unchecked in longer traces. We therefore introduce Continual Search, an iterative framework that nudges the judge over successive turns to keep looking for unresolved diagnostic evidence. We evaluate Continual Search on four existing RCA benchmarks and, recognizing the lack of large‑scale traces in current suites, we create MegaRCA‑Mix, a challenging testbed containing 50 human‑annotated failure trials from long‑horizon, execution‑heavy tasks. Across multiple benchmark suites and model families, Continual Search consistently improves attribution performance. On MegaRCA‑Mix, for instance, GPT‑5.5’s F1 score rises from $0.349$ to $0.498$, a gain of over 40%. Interestingly, within the same model family, lower‑tier models can even surpass higher‑tier ones when equipped with effective search, highlighting that search strategy outweighs raw model scale.
Review