Real‑world tasks are often complex and noisy, posing serious challenges for large language model (LLM) agents. Existing evaluations lack a systematic framework that jointly characterizes perturbation targets and types. We introduce a two‑dimensional view that models both aspects together, and use it to conduct a unified study that quantifies the robustness of frontier models, traces the mechanisms behind perturbation‑induced failures, and derives robustness‑oriented training recommendations.
Controlled evaluations on APEX‑Agents and Toolathlon show that all perturbations consistently degrade performance, while the dominant vulnerabilities differ across task environments. Trajectory‑level analysis reveals that agents rarely recognize altered inputs and often propagate corrupted information without verification. A focused cost analysis further demonstrates that successful perturbed trajectories can require substantially more computation.
These findings motivate training objectives that cover diverse noise forms, reinforce result verification, enhance uncertainty awareness, and improve recovery capabilities. We also provide a rigorous benchmark‑construction pipeline, with code and data to be released at https://github.com/xuyj233/AgentPerturb.
Review: The paper offers a systematic perturbation framework that uncovers critical weaknesses of LLM agents during interaction, offering clear guidance for future robustness‑focused training.