AI agents interact with external environments by invoking tools, and the Model Context Protocol (MCP) has become the standard interface for linking agents to heterogeneous tools. As organizations deploy agents at scale, MCP gateways serve as a critical middleware layer for aggregating, governing, and shaping the tool surface exposed to agents. Prior work has not systematically examined how gateway design influences agent capability, efficiency, and security. To fill this gap, we introduce GatewayBench—the first unified evaluation framework and benchmark for MCP gateways. We categorize gateway architectures into two major paradigms: flattened and hierarchical. The benchmark comprises 524 tasks across six domains and a pool of 11,118 real‑world tools drawn from 251 authentic MCP servers. To emulate large‑scale deployments, we inject task‑irrelevant tools sampled from this pool into each agent environment, enabling controlled scaling of the accessible tool surface. Using GatewayBench, we conduct extensive experiments on seven widely deployed MCP gateways and diverse agent configurations, covering both benign and adversarial tasks. Results reveal substantial differences among gateway designs in task completion rate, tool‑use efficiency, and resilience to attacks, and these effects interact strongly with the underlying model and agent implementation. The study highlights the MCP gateway as an important yet underexplored component of the agentic AI stack. GatewayBench provides practical guidance and a rigorous testbed for controlled evaluation of gateway architectures within realistic tool ecosystems, supporting the development of more capable, efficient, and secure agentic systems.
Review