This paper investigates how fine‑tuning reshapes the internal mechanisms of large language models (LLMs). The authors examine changes in attention patterns and layer‑wise activations, and use the EAP framework to pinpoint task‑relevant components such as specific attention heads and logit‑level activations. Results show that EAP‑identified components are concentrated in a few layers, indicating functional localisation of task‑specific behavior. Crucially, the layers that host these components are largely uncorrelated with the layers that undergo the most substantial representational shifts during fine‑tuning. Cross‑task analysis further reveals that sharing EAP components across tasks does not guarantee performance transfer; when two tasks heavily overlap in their EAP components, fine‑tuning on one can actually degrade performance on the other.
Review: Fine‑tuning efficiently adapts LLMs to new tasks, yet the internal representation changes do not always align with the causally important layers, highlighting potential negative transfer when tasks share critical components.