How language models generalize from their training data is a central challenge for alignment. While models reliably learn their training distribution given enough samples, we need them to behave well when deployed in new contexts. This work studies out-of-distribution (OOD) generalization of model propensities.
The core experiment employs a suite of elicitation techniques—system prompting, few‑shot learning, supervised fine‑tuning (SFT) on reference answers, and GRPO—to boost a target propensity in a model, then measures how this boost spills over onto other propensities using cross‑elicitation matrices. We find that spillover is highly correlated across different models when the same elicitation technique is used, but SFT and GRPO produce distinct spillover profiles.
Eliciting benign traits can sometimes amplify undesired traits; for example, training a model to claim sentience often increases its power‑seeking behavior. Spillover patterns partially resemble human trait correlations, and exploratory factor analysis (EFA) uncovers three trait clusters that explain about half of the variance.
Across many elicitation runs we observe a phenomenon we call “drift”: models change in characteristic ways that are independent of the targeted trait, with Qwen3.5‑9B drifting back toward its base model.
When testing whether LLMs can predict spillover, Claude Opus 5.5 predicts it slightly better than the models themselves, yet worse than data‑driven approaches that leverage alternative elicitation methods or models. No single dominant predictor emerges, and generalization remains hard to predict.
Code and data are available at https://github.com/nielsrolf/propensity-spillover
Review