Large language model agents are increasingly deployed via the Model Context Protocol (MCP), which lets them discover and invoke tools across multiple servers at runtime. Existing MCP benchmarks mainly measure task completion and diagnose failures after execution, without embedding specific reasoning, execution, and memory risk factors into task design. We introduce MCPHallu, a benchmark that diagnoses hallucination‑prone failures along three axes: reasoning, execution, and memory. MCPHallu comprises 358 tasks spanning five capability domains and runs on 36 production MCP servers in isolated containers. Each task includes a structural trigger that makes one of four subtypes the dominant failure risk—Branch Collapse, Unreachable Goal, Tool Misuse, or Context Forgetting—enabling subtype‑level analysis rather than aggregate success rates alone. We evaluated fourteen frontier LLM agents from eight vendors over 5,012 trajectories. Current MCP agents remain vulnerable to hallucination‑inducing tool environments, achieving an average score of 0.642 on a $[0,1]$ scale (higher is better). Failures concentrate on Unreachable Goal tasks, where the average score drops to 0.458 and only 23.2% of runs pass, indicating agents often continue issuing tool calls or claim success without recognizing infeasible goals. Subtype rankings diverge sharply: a model’s rank shifts by an average of 6.9 out of 13 positions across subtypes, and Branch Collapse is essentially uncorrelated with Unreachable Goal ($r = -0.12$). MCPHallu is released at https://github.com/J1zz/mcp_hallu.
Review: MCPHallu’s structured triggers break down hallucination risks into measurable subtypes, offering a finer‑grained diagnostic for MCP agents. The benchmark highlights a pronounced weakness in recognizing unattainable goals, pointing to a critical area for future model and tool‑integration improvements.