This study investigates whether small language models can detect and report on perturbations in their internal activations. Specifically, we explore this through activation steering by injecting concept vectors into the model's residual stream and measuring the model's accuracy in reporting these perturbations. We first highlight that the binary detection paradigm used in previous work—prompting the model to answer 'Yes' or 'No' regarding the detection of injected thoughts—confounds small models, as steering biases towards affirmative responses regardless of question content. Thus, we propose two confound-free evaluation paradigms: sentence localization (identifying which of $N$ sentences was perturbed, chance $= 1/N$) and strength comparison (determining which of two sentences received a stronger injection, chance $= 50 ext{ extbackslash%}$).
Evaluating across six models from two families (Llama-3.2 and Gemma-4), we find that models as small as 2B parameters introspect reliably above chance, with introspective ability generally increasing with scale. However, Llama-1B performs at or below chance. We then introduce Introspection Fine-Tuning (IFT): supervised fine-tuning on sentence-localization examples constructed from the model's own perturbed forward passes. IFT increases Llama-1B's sentence-localization accuracy from $9.6 ext{ extbackslash%}$ to $60.6 ext{ extbackslash%}$ (a $6 imes$ improvement), with gains generalizing zero-shot to the held-out strength-comparison task ($30.2 ext{ extbackslash%} o 52.2 ext{ extbackslash%}$). IFT also enhances introspection for 3B and 8B models while inducing negligible degradation on standard capability benchmarks. Our findings suggest that introspective ability is not solely determined by scale: it can be directly trained, unlocking latent self-monitoring capacity with implications for AI transparency and alignment. Our code is available here.
Blogger's Review: The introduction of Introspection Fine-Tuning (IFT) significantly enhances the self-monitoring capabilities of small language models, providing new avenues for AI transparency and alignment. This innovative training method may boost the performance of small models in complex tasks, making it a noteworthy development in the field.