Advances in large language models have spurred research on autonomous agents, focusing on perception, memory, reasoning, planning, and action. With the rise of large multimodal models, agents can ingest images, audio, and video, greatly expanding real‑world applicability.
The survey first traces the shift from text‑only agents to multimodal frameworks, highlighting three integration patterns—delegated, late‑fusion, and early‑fusion—that feed multimodal signals into an LLM backbone. It then examines how grounded perception enables visual and auditory reasoning and planning, empowering agents to perform tasks such as robot manipulation, GUI/web navigation, multimedia generation, and long‑form video understanding.
Using a modality‑centric taxonomy, existing works are mapped to architectural choices and capability profiles, and performance, training/inference cost, latency, and deployment constraints are compared across application domains.
The authors point out significant gaps in cross‑modal coordination, long‑term memory retention, and generality, and outline a roadmap toward robust, general‑purpose intelligent systems.
Blogger's Review: This survey offers a comprehensive synthesis of multimodal agentic techniques, providing a clear framework and future directions for researchers in the field.