NeFut Logo NeFut
Admin Login

[CS.AI] How Many Thoughts Can a Vector Hold? The Capacity of Reasoning by Superposition

Published at: 2026-09-15 22:00 Last updated: 2026-09-16 00:22
#AI #Machine Learning #LLM

Large language models solve complex problems by performing intermediate computations across multiple reasoning steps. Traditional chain‑of‑thought serializes these computations as tokens. Recent continuous and recurrent approaches move partial results into fixed‑dimensional latent states, allowing a single “thought” to superpose several alternatives.\ \ A fundamental design question is what continuous thoughts should preserve as reasoning proceeds. An intuitive strategy discards past computations and keeps only the current reasoning frontier, assuming that storing more would dilute the state and waste the limited representational budget.\ \ We show that this intuition can be wrong. When downstream computations are identical, a cumulative superposition that retains the full reasoning history can require fewer representational dimensions than a frontier‑only superposition. With a fixed hidden width, this advantage lets latent reasoners keep more valid evidence, distinguish a larger set of plausible downstream outcomes, and postpone the point at which compressed states become unreliable.\ \ The counter‑intuitive effect arises because informative historical components coherently reinforce each other, while unrelated alternatives introduce random interference. This perspective also answers a practical design question: how should a model weight memories accumulated inside latent states when their future use is unknown? In reusable weighted superpositions, prioritizing a small set of recent or salient items yields weakly represented memories that become bottlenecks for subsequent attention. Uniform cumulative weighting avoids this flaw, and we prove it is minimax‑optimal for robust future reasoning.\ \ Thus, superposition is not merely an observed latent‑space phenomenon but a design principle: balanced cumulative memory enables a fixed representational budget to support more reliable, reusable computations.\ \ Review: Overall, cumulative superposition offers an unexpected advantage in preserving historical information, suggesting a new direction for designing latent‑space reasoning in large models.

Original Source: https://arxiv.org/abs/2609.13747

[h] Back to Home