Biological intelligence continuously incorporates new information while preserving prior skills through localized plasticity. In contrast, continual learning in large language models (LLMs) still suffers from catastrophic forgetting. A promising approach is to identify functionally specialized subnetworks within LLMs, enabling targeted and resource‑efficient adaptation. By applying gradient‑attribution profiling, we locate cognition‑related functional subnetworks that exhibit extremely low mutual overlap. Building on this insight, we introduce Function‑isolated Memory‑Efficient Optimization (CiMEO): each training example is routed to its corresponding functional subnetwork, and parameter updates as well as optimizer states are confined to the selected region.
Across 1.4 million sequential instructions, CiMEO attains the highest average performance among compared methods on five generative benchmarks covering programming, exam‑style tasks, and mathematical reasoning, while markedly reducing catastrophic forgetting. Subspace‑overlap and principal‑angle analyses further show that localized updates help preserve pre‑trained representations, highlighting functional subnetwork localization as a practical strategy for efficient and stable continual learning in LLMs.
Review: CiMEO’s fine‑grained subnetwork isolation achieves new levels of learning efficiency without erasing existing knowledge, offering a scalable solution for the long‑term adaptability of large models.