Activation steering adjusts LLM behavior at inference by adding learned directions to hidden states, but existing approaches handle a single concept at a time. Pluralistic alignment demands simultaneous emphasis on several value dimensions, for example when different stakeholders prioritize distinct aspects of the same topic. Naïvely adding multiple directions leads to substantial spillover: the effect intended for one value leaks into others. This mirrors the treatment‑versus‑spillover decomposition in causal inference.
We trace spillover to geometric entanglement of steering directions, captured by the Gram matrix $G = D^{\top}D$ of the direction matrix $D$. Non‑diagonal entries indicate mixing between values. To decouple each direction’s contribution we introduce an activation‑norm‑penalized objective:
$$\mathcal{L}_{\text{norm}} = \|h + D\alpha\|_2^2 - \lambda \|\alpha\|_2^2,$$
where $h$ is the original hidden state, $\alpha$ the coefficient vector, and $\lambda$ a regularization weight. Optimizing this yields $\alpha$ that preserves the overall norm while making each direction’s effect independent, achieving a zero‑cost spillover correction.
An end‑to‑end pipeline follows: given only a set of domain questions, the system automatically discovers value dimensions, extracts directions, diagnoses non‑diagonal Gram components, and applies the corrected steering, without any fine‑tuning, reward model, or manual prompt engineering. On climate‑discourse data the correction raises net steering impact from +5.9% to +14.0%, validated over 100,000 pairwise judgments.
Review