During prolonged wakefulness, mammalian neocortical networks exhibit brief, localized neuronal silences called off‑states. This work analogizes such spatiotemporally concentrated disruptions to the sequential reasoning process of autoregressive large language models (LLMs) and measures their impact on performance.
We evaluate 250 paired derivations on the GSM8K benchmark using two 8‑billion‑parameter models (Qwen3‑8B and LLaMA‑3.1‑8B‑Instruct). Two dropout strategies share the same total budget:
- Concentrated burst dropout: high‑density masking of attention heads in a few generation steps;
- Diffuse i.i.d. dropout: uniform random masking across all steps.
At a budget of $\mathcal{B}=0.10$ (approximately $5.7\%$ of vulnerable head‑token volume), Qwen3‑8B accuracy falls from $91.2\%$ under diffuse noise to $78.8\%$ with burst dropout (Δ = ‑12.4 pp, McNemar $p=5.5\times10^{-6}$), while LLaMA‑3.1‑8B‑Instruct drops from $84.0\%$ to $76.8\%$ (Δ = ‑7.2 pp, $p=0.0198$). The same budget applied diffusely yields only mild degradation, preserving the original scores.
A factorial follow‑up experiment ($N=100$) shows that single‑step dropout density correlates with an acute accuracy drop in a single schedule draw (Qwen ‑10.0 pp, $p_{\text{holm}}=0.0425$), whereas multi‑token continuity exhibits an inconclusive compounding trend (‑7.0 pp, $p_{\text{holm}}=0.1892$).
Error analysis reveals that Qwen failures concentrate in formatted answer derivations (conditional accuracy loss of $11.4$ pp), whereas $22.2\%$ of LLaMA errors lack a final answer line, narrowing the conditional gap to $5.0$ pp.
Conclusion: Autoregressive reasoning chains are especially vulnerable to highly concentrated structural disruptions, providing empirical benchmarks for fault‑tolerance in transformer architectures.
Review: Concentrated attention‑head masking dramatically impairs multi‑step reasoning, suggesting that real‑world deployments should guard against sudden, localized hardware or software failures.