Continual fine‑tuning is essential for large language models (LLMs) to adapt dynamically in real‑world settings, yet it inevitably causes catastrophic forgetting, manifested as degradation of previous tasks and loss of the model’s general‑purpose knowledge. Existing techniques such as orthogonal gradient projection alleviate inter‑task forgetting but fail to preserve the inherent knowledge from pre‑training because they require the original data and gradients, which are unavailable and highly diverse for off‑the‑shelf LLMs. To close this gap we introduce EoupCT (Estimate and Orthogonalize Unknown Pre‑training Gradients for Continual LLM fine‑Tuning). The framework works as follows: 1) A learnable soft prompt equipped with Gumbel‑Softmax relaxation generates pseudo data that is most vulnerable to forgetting for the new task, thereby estimating the unknown pre‑training gradients; 2) The task loss, new‑task gradients, and estimated pre‑training gradients are formulated as a multi‑objective optimization problem; 3) A first‑order efficient Pareto optimizer jointly updates the LLM parameters and the soft prompt while enforcing strict orthogonality between new‑task updates and the estimated pre‑training gradients. Extensive experiments on various LLMs show that EoupCT preserves both task‑specific proficiency and the model’s general knowledge, effectively mitigating catastrophic forgetting.
Review