Activation steering has recently been employed as a memory mechanism for skills: past experiences are encoded as steering vectors and later retrieved. The common assumption is that user facts and preferences must be stored as text. We attach a training‑free activation memory next to a frozen LLM to test this assumption. After an interaction we record the change in the model’s late‑layer states, treat it as a shift, file it under a whitened key representing the conversation’s gist, and re‑inject the shift in a later session only when the gist matches.
We argue that such a shift is a fixed average of the effect that attention recomputes for each question in context, so only question‑independent content can survive. In a pre‑registered study on Qwen3.5‑2B and 9B, with judges from four model families, the memory proved highly selective—about 98% of unrelated replies remained unchanged. However, facts are stored mainly as the word to say in the last layers, and are almost never used (at most 15% clean use, versus 96‑98% when provided directly in context).
Memory transfer occurs only within a narrow strength window. Strikingly, the reference subtracted when writing a preference decides whether the concept or its direction survives: unless the opposite reference is subtracted, a stored dislike is recalled as a like.
Review: The core of activation‑steering memory is a gist‑driven state shift that can only retain abstract, question‑agnostic information, and it stores factual knowledge extremely sparsely. Practical deployment must carefully tune shift magnitude and reference subtraction, otherwise preference signals may flip direction.