LLM agents interact with web services to accomplish tasks on behalf of users, but errors frequently arise during these interactions, causing execution stalls or task failures. The conventional response is to intervene immediately, yet agents often recover on their own in subsequent steps, raising a crucial question: when should an external system intervene and when should it wait for self‑recovery?
To answer this, we built ReInBench, a benchmark containing 1,259 agent execution traces and 1,403 error events across both local‑compute and web‑based tasks. Using this dataset, we examined three key aspects:
- Frequency and latency of self‑recovery. About 725 errors (≈52%) recover within a few steps, and recovery speed varies significantly across different execution LLMs.
- Impact of self‑recovery on subsequent execution. Self‑recovery not only raises the success rate of later errors but also improves overall task completion.
- Predictability of self‑recovery at the moment an error occurs. Error type, contextual cues, and the characteristics of the execution LLM all contribute to reliable prediction.
Guided by these insights, we propose ReCoF (Recovery‑oriented Classification Framework), a method that predicts whether an error will self‑recover and thus whether additional intervention is required. ReCoF achieves a 9.02% average relative improvement in Macro‑F1 over the strongest baseline on ReInBench. The code and data are released at https://method-notes.github.io/ReCoF/.
Review: This work provides a large‑scale empirical foundation that quantifies the prevalence and limits of self‑recovery in LLM agents, offering a data‑driven basis for smarter intervention policies. The ReCoF predictor demonstrates a practical path toward “intervene only when necessary,” paving the way for more autonomous and reliable task‑execution systems.