Latent reasoning methods replace verbose explicit chain‑of‑thought (CoT) tokens with compact continuous‑space embeddings, improving token‑level efficiency and robustness. Existing approaches lack direct process supervision for these latent embeddings, which can cause representation collapse and uneven information distribution. To remedy this, we introduce Prototype‑Mediated Process Supervision (PMPS), adding learnable reasoning prototypes as semantic anchors to provide structural, process‑level guidance for latent reasoning. PMPS projects both latent embeddings and explicit CoT embeddings into a shared prototype space, achieving many‑to‑many soft alignment between unequal‑length representations via prototype assignment. We also propose a Progressive Sequential Alignment (PSA) module: positional priors initially encourage a sequential alignment structure, then gradually relax to allow adaptive matching. Experiments on GSM8K‑Aug show that PMPS compresses output token length to below 50% of explicit CoT. Compared with the strong baseline SIM‑CoT, it gains an average 2.08% accuracy improvement and even surpasses CoT‑SFT on GPT‑2. On larger models and more challenging tasks, PMPS consistently attains the highest accuracy among latent reasoning methods while keeping output length comparable.
Review