NeFut Logo NeFut
Admin Login

[CS.AI] Function Lives Where Variance Doesn't: Task-Weighted Charts of a Language Model's Computation

Published at: 2026-09-18 22:00 Last updated: 2026-09-20 12:54
#optimization #LLM #Artificial Intelligence

How many dimensions does a language model actually use in its computation? Without naming a functional, the question is ill‑posed. Task‑weighted charts resolve this by fitting a low‑dimensional coordinate system to the representation under the metric induced by a chosen functional, turning distillation into ordinary least‑squares regression.\ \ Experiments on six models ranging from 70 M to 7 B parameters show that preserving next‑token perplexity within 5% requires retaining 70%‑90% of the residual‑stream width. This width is consumed by the rare tail of language, and the variance profile fails to predict it: two directions carry 90% of GPT‑2’s activation variance yet contribute almost nothing to its function. Dimension must be defined per‑functional—model uncertainty can be captured by six coordinates, whereas the full predictive distribution needs hundreds, and it grows with depth.\ \ The dissociation is exploitable: when only a few dimensions can be kept, charts trained under the functional’s metric preserve predictions better than variance‑based or optimal linear compression.\ \ Review: Task‑weighted charts offer a function‑centric low‑dimensional view of language‑model computation, revealing that the most “important” directions are not those with highest variance, which has significant implications for model compression and interpretability.

Original Source: https://arxiv.org/abs/2609.18989

[h] Back to Home