Comprehensive video understanding is essential for advancing AI toward the intricate dynamics of the physical world. Recent Multimodal Large Language Models (MLLMs) have shown strong video comprehension abilities, yet existing benchmarks are confined to single‑step scene queries or global summaries, lacking the multi‑hop multimodal reasoning required in real‑world scenarios. To address this gap we introduce AgentVidBench, a multi‑hop video question answering benchmark that targets spatial, temporal, and causal reasoning of MLLM agents. In addition to standard QA pairs, the benchmark supplies step‑by‑step solution traces, enabling evaluation of whether an agent explicitly gathers the evidence needed to justify its answer. Experiments with twelve proprietary and open‑source MLLMs reveal that single‑turn performance remains limited, while integrating these models into state‑of‑the‑art agentic pipelines generally improves both accuracy and trajectory scores. We also present a simple yet effective agentic strategy that serves as a competitive baseline, establishing AgentVidBench as a holistic testbed for future research on agentic video understanding. Code and datasets are released at https://github.com/krafton-ai/agentvidbench and https://huggingface.co/datasets/agentvidbench/agentvidbench.
Review