Full fine‑tuning achieves strong results on reasoning‑intensive tasks but demands heavy GPU consumption. Parameter‑efficient fine‑tuning (PEFT) reduces this cost, yet a noticeable performance gap remains on such tasks. Our three key observations reveal that the gap stems from norm‑related under‑adaptation: PEFT yields markedly smaller weight‑update norms compared with full fine‑tuning. To address this, we introduce NormTune, a norm‑guided PEFT strategy that adaptively amplifies updates for layers with insufficient weight‑update norms. Extensive experiments demonstrate its robustness across challenging reasoning benchmarks. Trained on S1K-1.1, integrating NormTune into rank‑8 LoRA, Bone, and MoRA improves average accuracy by 24.24%, 11.33%, and 16.21% respectively on AIME 2024, MATH 500, GPQA, and AIME 2025. Moreover, NormTune‑enhanced rank‑256 LoRA, Bone, and MoRA achieve performance comparable to S1 full fine‑tuning, with average gains of 2.17%, 0.93%, and 3.00% over the baseline.
Review: NormTune’s simple norm‑scaling mechanism effectively bridges the adaptation gap of PEFT in reasoning tasks, offering a practical solution for high‑quality fine‑tuning under limited computational resources.