NeFut Logo NeFut
Admin Login

[CS.AI] Interpreting Large Language Models at Scale

Published at: 2026-08-12 22:00 Last updated: 2026-08-13 01:53
#Machine Learning #LLM #Open Source #Artificial Intelligence #GPT

A recent study proposes a method called OmniLens for interpreting large language models (LLMs). This method maps intermediate activations to the output vocabulary, revealing how next-token predictions develop through the network. However, trained lens parameters grow quadratically with model width, while exact, full-vocabulary Kullback-Leibler (KL) training dominates memory. As a result, previous studies have been limited to models with at most 20B parameters and are tied to specific component types. OmniLens applies a single lens family to any model-width activation, whether residual stream, attention, or MLP, and combines two independent scaling techniques: low-rank translators and Subset-KL. Low-rank translators make per-lens parameter growth linear in model width and reduce trainable parameters by up to 98.4%. Subset-KL materializes only selected vocabulary logits: its Top-k mode cuts peak training memory by up to 70%, while its importance-sampled variant retains unbiased stochastic gradients for the full KL. These savings enable a dense ensemble of 482 lenses for LLaMA-3.3-70B, providing 6x the coverage of a residual-stream design at the same depth. Model-wide coverage reveals what single-component lenses cannot: the components where a behavior is most visible need not be those where intervention is most effective, and the most effective interventions lie outside the attention heads examined by prior lens studies. Across three case studies (prompt-injection detection, multi-hop memory injection, and toxicity localization), OmniLens reproduces key published results at substantially lower cost. Blogger's Review: The introduction of OmniLens marks a significant breakthrough in the field of large language model interpretation, enabling us to better understand the internal workings of these models and providing new insights and methods for future model optimization and applications.

Original Source: https://arxiv.org/abs/2608.10260

[h] Back to Home