We introduce the Self-Verified Reasoner (SVR-R1), a multi-turn reinforcement learning framework that turns a model's own verification into a learning signal for multimodal reasoning. For each query, the model proposes an answer using the same weights and issues a binary self-verdict (Yes/No). A 'No' triggers a second-chance rethink, while a 'Yes' or a turn cap finalizes the output for computing the outcome-based reward.
SVR-R1 is implemented with GRPO and an asynchronous multi-turn rollout framework, requiring no external supervision or auxiliary critics. We evaluate SVR-R1 on vision-language reasoning benchmarks and show that it improves accuracy by a large margin over strong standard GRPO baselines. The training dynamics reveal a decreasing reliance on verification—with fewer verification turns but higher test accuracy—indicating that the gap between verification and generation narrows as the policy internalizes self-correction and chooses the most confident answer through our framework.
SVR-R1 bridges the less explored intersection of inference-time self-refinement and RL training for VLMs, offering a simple yet effective recipe for bootstrapping multimodal reasoning. We will open-source SVR-R1 to facilitate future research in VLMs.
Blogger's Review: SVR-R1 significantly enhances multimodal reasoning performance through its self-verification mechanism, showcasing the potential of reinforcement learning in an unsupervised setting. The shift in training dynamics indicates that the model is gradually mastering self-correction capabilities, paving the way for new directions in future research, which is worth watching.