NeFut Logo NeFut
中 Admin Login

[CS.AI] EGI-BENCH: An Enterprise General Intelligence Benchmark Suite

Published at: 2026-10-21 22:00 Last updated: 2026-10-22 01:04
#AI #LLM #GPT

Enterprise adoption of large language models (LLMs) in agentic workflows requires evaluations across a broad spectrum of abilities: CRM operations, heterogeneous‑document retrieval, personal data understanding, multi‑turn planning, tool use, function calling, long‑horizon research, and long‑context software engineering. Currently these evaluations reside in separate repositories with divergent runners, dependencies, and scoring conventions, making cross‑benchmark comparison difficult and obscuring the meaning of a single aggregate score. We release EGI‑BENCH, a unified LLM agent benchmark suite that consolidates eight enterprise‑facing agent benchmarks (CRMARENA, HERB, PERSONABENCH, USERBENCH, MCP‑UNIVERSE, MCPEVAL, BENCHFORCE, LOCOBENCH) behind a single command‑line interface with two runtime‑budgeted execution modes: quick and deep. On top of the suite we define the Enterprise General Intelligence (EGI) Score, a six‑dimensional framework (M1‑M6) whose buckets are populated by fine‑grained sub‑metrics routed from each benchmark rather than by benchmark‑level averages. We describe the suite architecture, sub‑metric routing, and the quick/deep budgets, and report results for fourteen frontier and cost‑efficient LLMs spanning the GPT, Claude, and Gemini families under quick mode. Top‑line EGI ranges from 39.6 (GPT‑4o mini) to 65.6 (gpt‑5.5), and no single model dominates every capability axis: gpt‑5.5 leads task outcome, planning, and retrieval grounding; Claude Sonnet 4.5 leads tool actioning and safety/abstention; Claude Haiku 4.5 leads engineering quality. Code is available at https://github.com/SalesforceAIResearch/egi-bench

Review

Original Source: https://arxiv.org/abs/2610.27234

[h] Back to Home