缓解灾难性遗忘的动态正交持续微调
Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
AI总结：
针对 LLM 持续微调中的灾难性遗忘，论文提出 DOC，通过动态跟踪历史功能方向并令新任务梯度与其正交来减少新旧任务干扰，实验证明其优于现有方法。
摘要：
灾难性遗忘仍然是大语言模型（LLM）持续学习中的一项关键挑战：在无法访问过去数据集的情况下，模型基于新的序列数据进行微调时，难以保持在历史任务上的性能。本文首先揭示，微调过程中功能方向的漂移，是现有基于正则化的方法在长期 LLM 持续学习中失效的关键原因。为解决这一问题，我们提出动态正交持续（Dynamic Orthogonal Continual，DOC）微调，这是一种新颖方法，可在微调过程中跟踪这些功能方向的漂移并对其进行动态更新。此外，通过调整新任务参数的梯度，使其与所跟踪的历史功能方向正交，我们的方法缓解了新任务与旧任务之间的干扰。在多种 LLM 持续学习基准上进行的大量实验表明，该方法优于先前方法，能够有效减少灾难性遗忘，并为持续 LLM 微调提供一种鲁棒工具。我们的代码可在 https://github.com/meloxxxxxx/DOC 获取。
英文摘要：
Catastrophic forgetting remains a critical challenge in continual learning for large language models (LLMs), where models struggle to retain performance on historical tasks when fine-tuning on new sequential data without access to past datasets. In this paper, we first reveal that the drift of functional directions during the fine-tuning process is a key reason why existing regularization-based methods fail in long-term LLM continual learning. To address this, we propose Dynamic Orthogonal Continual (DOC) fine-tuning, a novel approach that tracks the drift of these functional directions and dynamically updates them during the fine-tuning process. Furthermore, by adjusting the gradients of new task parameters to be orthogonal to the tracked historical function directions, our method mitigates interference between new and old tasks. Extensive experiments on various LLM continual learning benchmarks demonstrate that this approach outperforms prior methods, effectively reducing catastrophic forgetting and providing a robust tool for continuous LLM fine-tuning. Our code is available at https://github.com/meloxxxxxx/DOC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
