发表机构
Jilin University; RIKEN Center for Advanced Intelligence Project; University of Warwick; University of Tokyo（吉林大学; 理化学研究所先进智能项目中心; 华威大学; 东京大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对持续微调中的灾难性遗忘，提出EoupCT框架，通过动态生成伪数据估计未知预训练梯度，并利用Pareto优化器强制正交性，在多个LLM上有效保留任务能力与通用知识。
AI 中文摘要
持续微调对于大型语言模型（LLMs）动态适应现实环境至关重要，然而它不可避免地遭受灾难性遗忘，特别是先前任务性能的下降以及LLMs通用知识的退化。尽管现有方法（如正交梯度投影）缓解了各种微调任务中的遗忘，但它们从根本上无法保留预训练LLMs固有的通用知识，因为这些方法所需的现成预训练LLMs的原始数据和梯度是严格未知且高度多样化的。为弥合这一关键差距，我们提出了EoupCT，一个新颖的框架，旨在为持续LLM微调估计并正交化未知的预训练梯度。具体而言，EoupCT通过一个配备Gumbel-Softmax松弛的可学习软提示，动态生成对新任务最易遗忘的伪数据来估计预训练梯度。此外，我们制定了一个多目标优化问题，并引入了一个一阶高效的Pareto优化器，该优化器联合优化LLM参数和软提示，严格强制新任务更新与估计的预训练梯度之间的正交性。跨多个LLM的大量实验表明，EoupCT有效保留了任务特定熟练度和固有的通用知识，成功缓解了灾难性遗忘。
英文摘要
Continual fine-tuning is essential for large language models (LLMs) to dynamically adapt to real-world environments, yet it inevitably suffers from catastrophic forgetting, particularly the performance degradation of previous tasks and LLMs' general-purpose knowledge. Although existing methods, such as orthogonal gradient projection, mitigate the forgetting across various fine-tuning tasks, they fundamentally fail to preserve pre-training LLMs' inherent general-purpose knowledge because the original data and gradients of off-the-shelf pre-training LLMs required by these methods are strictly unknown and highly diverse. To bridge this critical gap, we propose EoupCT, a novel framework designed to Estimate and Orthogonalize Unknown Pre-training gradients for Continual LLM fine-Tuning. Specifically, EoupCT estimates pre-training gradients by dynamically generating pseudo data that is most susceptible to forgetting for new tasks through a learnable soft prompt equipped with Gumbel-Softmax relaxation. Furthermore, we formulate a multi-objective optimization problem and introduce a first-order efficient Pareto optimizer that jointly optimizes LLM parameters and the soft prompt, rigorously enforcing orthogonality between new task updates and the estimated pre-training gradients. Extensive experiments across multiple LLMs demonstrate that EoupCT effectively preserves both task-specific proficiency and inherent general-purpose knowledge, successfully mitigating the catastrophic forgetting.
CommentsAccepted by NeurIPS 2026. 29 pages, 3 figures. Code: https://github.com/wangbing1416/EoupCT
