Reinforcement learning with verifiable rewards has become the dominant approach for eliciting test-time scaling in explicit Chain-of-Thought (CoT) reasoners. However, this scaling path remains computationally expensive, as every intermediate step must be decoded as a language token.
In contrast, latent reasoning carries intermediate computation as continuous vectors and already matches or exceeds explicit CoT at much shorter horizons. Despite this promise, latent reasoners remain largely imitation-bound, while explicit CoT has advanced beyond imitation through outcome-reward RL.
Latent trajectories lack a tractable per-step likelihood and an adaptive stopping interface under fixed thinking budgets, making it difficult for outcome rewards to drive latent test-time scaling.
We introduce Surrogate Latent Policy Optimization (SLPO) to integrate outcome-reward RL into autoregressive latent reasoners: an empirical surrogate policy density over latent transitions for trajectory-level credit assignment, and a correctness-supervised stopping head that refines outcome-reward optimization into a variable-horizon policy.
Across continuous and soft thinking settings, SLPO improves Pass@$k$ under parallel sampling and allocates longer latent computation to harder instances with higher deterministic accuracy.
Blogger's Review: The introduction of SLPO offers a novel perspective and method for latent reasoning, especially in balancing computational efficiency and accuracy. By employing effective surrogate policies, SLPO not only addresses the shortcomings of existing latent reasoning approaches but also provides a direction for future research. Its potential to enhance the handling of complex instances is particularly noteworthy.