Human vision operates as a closed loop: gaze is continuously redirected by intermediate hypotheses rather than a single snapshot. Decades of psychophysics and cognitive science suggest that this active observation is crucial for a variety of tasks. However, whether today's multimodal large language models (MLLMs) exhibit active observation remains an empirical question that existing vision-language benchmarks fail to address.
To tackle this, we introduce ActiveVision, a benchmark that measures active observation in MLLMs, comprising 17 tasks across 3 categories. These tasks are designed to compel repeated visual perception rather than a single static description. Current frontier MLLMs struggle with ActiveVision: the highest-scoring model we evaluated, GPT-5.5, at the top reasoning-effort tier, solves only 10.6% of items, scoring zero on 11 out of 17 tasks. Claude Fable 5, despite leading in many reasoning and coding leaderboards, solves only 3.5%, significantly trailing three human participants who average 96.1%.
Moreover, much of the gap persists even when models write and run their own vision code: such code proves unreliable on realistic imagery, and catching its failures requires the very active perception that the models lack. Together, these findings indicate that current MLLMs lack robust active visual observation, highlighting the need for architectures and training objectives that bridge the perception-reasoning loop.
Blogger's Review: This research underscores a significant shortcoming in MLLMs regarding visual understanding, emphasizing the importance of active observation in cognition. Future model designs should focus on integrating perception and reasoning more closely to enhance performance on complex tasks.