Visual agents that intertwine reasoning with tool invocation show promise for multimodal tasks. Traditionally, learning effective tool use relies on large-scale QA pairs or expert demonstrations, which are costly. We therefore introduce Marvo, a simple yet effective self‑evolution method that learns from unlabeled images. Marvo consists of two entities: a challenger that creates QA pairs from raw images, and a solver that learns tool invocation by solving the questions via reinforcement learning. To encourage tool use, the challenger receives a margin reward defined as the solver's performance gap with and without tools. This margin reward yields an adaptive curriculum, ensuring question difficulty is just enough to drive tool learning. Experiments across benchmarks covering agentic reasoning, visual search, mathematical reasoning, and real‑world reasoning show Marvo improves average accuracy by up to 7.1%. The results demonstrate margin reward as the preferred way to scale question complexity without human supervision.
Review