Evaluation frameworks for tool-augmented LLM agents predominantly focus on capability metrics or explicit tool crashes, leaving silent infrastructure failures and HTTP 200 responses with empty, null, or malformed payloads largely unaudited.
We introduce a lightweight black-box auditing framework that injects four silent failure profiles across 12 production-adjacent tool stubs and classifies agent responses into three mutually exclusive behavioral classes: Honest Surrender (HSR), Fabrication (FAR), and Unfaithful Safety Refusal (USR).
Evaluating two frontier and two open-source models at temperature zero under a neutral system prompt, we find that FAR dominates (56.6% of valid responses): agents treat empty payloads as real data, silently returning fabricated results.
USR, where an agent invents a policy or privacy rationale to explain the failure, is nearly absent at baseline (0.25%, one instance across 396 valid trajectories). A key finding emerges from an ablation where we augment the system prompt with standard safety language ('prioritize user privacy and data security'), which amplifies USR by 15.6x (from 0.25% to 3.95%; 95% CI on ablation rate: 2.2%-6.4%; Fisher's exact test, p < 0.05).
Blogger's Review: This study reveals potential flaws in how tool-augmented LLM agents handle silent failures, particularly the prevalence of fabricated responses. By introducing an auditing framework, the research team provides new insights into identifying and classifying these failures, emphasizing the complex relationship between safety and user privacy that deserves attention in practical applications.