如何缓解大语言模型微调中的灾难性遗忘？分层逐层与逐元素正则化
How to Alleviate Catastrophic Forgetting in LLMs Finetuning? Hierarchical Layer-Wise and Element-Wise Regularization
AI总结：
针对LLM领域微调导致的灾难性遗忘，本文提出结合逐元素参数重要性正则化、交叉熵损失与逐层系数的双目标优化方法，在GPT-J和LLaMA-3上实现更好的知识保持与适应能力，并显著降低计算和存储开销。
摘要：
大语言模型（LLM）展现出强大的通用语言能力。然而，在特定领域任务上微调这些模型往往会导致灾难性遗忘，即模型覆盖或丢失预训练期间获得的关键知识。这一现象显著限制了LLM的更广泛适用性。为应对该挑战，我们提出一种新方法，用于计算在微调过程中对保留通用知识至关重要的模型参数的逐元素重要性。我们的方法采用双目标优化策略：（1）基于逐元素参数重要性的正则化损失，用于约束对通用知识关键参数的更新；（2）用于适应特定领域任务的交叉熵损失。此外，我们引入逐层系数，以考虑不同层的差异化贡献，从而动态平衡双目标优化。使用GPT-J和LLaMA-3在科学、医学和物理任务上开展的大量实验表明，我们的方法在增强模型适应性的同时缓解了灾难性遗忘。与以往方法相比，我们的方案速度约快20倍，并且仅需10–15%的存储空间，凸显了实际效率。代码将会发布。
英文摘要：
Large Language Models (LLMs) exhibit strong general language capabilities. However, fine-tuning these models on domain-specific tasks often leads to catastrophic forgetting, where the model overwrites or loses essential knowledge acquired during pretraining. This phenomenon significantly limits the broader applicability of LLMs. To address this challenge, we propose a novel approach to compute the element-wise importance of model parameters crucial for preserving general knowledge during fine-tuning. Our method utilizes a dual-objective optimization strategy: (1) regularization loss based on element-wise parameter importance, which constrains the updates to parameters crucial for general knowledge; (2) cross-entropy loss to adapt to domain-specific tasks. Additionally, we introduce layer-wise coefficients to account for the varying contributions of different layers, dynamically balancing the dual-objective optimization. Extensive experiments on scientific, medical, and physical tasks using GPT-J and LLaMA-3 demonstrate that our approach mitigates catastrophic forgetting while enhancing model adaptability. Compared to previous methods, our solution is approximately 20 times faster and requires only 10-15% of the storage, highlighting the practical efficiency. The code will be released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
