NeFut Logo NeFut
中 Admin Login

[CS.AI] MMEnterprise-Bench: Evaluating Enterprise Agents over Evolving Multimodal Records

Published at: 2026-10-22 22:00 Last updated: 2026-10-25 13:04
#AI #Machine Learning #LLM

Enterprise agents must answer queries over workplace records that continuously evolve through team collaboration. Information is shared, corrected and revised across documents, chats and figures, resulting in multiple versions of the same fact. Reliable QA therefore requires agents to reason over sufficient evidence, resolve cross‑source and temporal conflicts, and detect when evidence is insufficient. To evaluate these capabilities systematically we introduce MMEnterprise‑Bench, which contains 1,613 questions grounded in evolving multimodal records from simulated collaboration projects and covers three categories: evidence reasoning, conflict resolution and unanswerability. We benchmark ten proprietary and open‑source multimodal LLMs and find that even the strongest models struggle when evidence involves cross‑source or temporal conflicts or spans several images. Increasing the retrieval budget or using a larger retriever improves evidence recall but does not consistently raise answer accuracy. These results highlight the persistent difficulty of reasoning over evolving workplace records and position MMEnterprise‑Bench as a testbed for building more reliable enterprise agents. Code and data are released publicly.

Review

Original Source: https://arxiv.org/abs/2610.27911

[h] Back to Home