通过模型增长缓解持续学习中的灾难性遗忘
Mitigating Catastrophic Forgetting in Continual Learning through Model Growth
AI总结：
本研究针对大语言模型持续学习中的灾难性遗忘问题，探索基于transformer堆叠的模型增长策略，实验发现该策略可适度缓解遗忘，但在社会偏见处理上存在权衡。
摘要：
灾难性遗忘是持续学习中的一个重大挑战，即模型在针对新任务进行微调时会丢失先前的知识。这一问题在经历持续学习的大语言模型（large language models, LLMs）中尤为关键，因为在不同领域保持性能对其通用效用至关重要。本文探讨了模型增长这一颇具前景的策略，该策略利用更小的模型来加速并组织更大模型的训练，以缓解灾难性遗忘问题。尽管基于增长的预训练（尤其是通过transformer堆叠实现的）已在加速收敛方面展现出潜力，但其对遗忘的影响仍未得到充分研究。因此，我们评估了基于增长的模型在一系列涉及领域知识、推理、阅读理解和偏见的微调任务中，是否能更有效地保留先前习得的能力。我们的研究结果表明，两种模型——采用增长训练的模型（Stack LLM）和未采用增长训练的模型（LLM）——在领域知识方面均有提升。然而，推理和阅读理解能力会随时间下降，显示出灾难性遗忘的迹象。Stack LLM始终表现出更小的性能下降，尤其是在阅读理解方面，表明其保留能力有所增强。有趣的是，在偏见评估中，基线LLM随着持续微调变得愈发中立，而Stack LLM则将偏见比例稳定维持在60%至61%左右。这些结果表明，基于增长的预训练可能在抵抗灾难性遗忘方面带来适度的提升，不过在处理社会偏见方面仍存在权衡。
英文摘要：
Catastrophic forgetting is a significant challenge in continual learning, in which a model loses prior knowledge when it is fine-tuned on new tasks. This problem is particularly critical for large language models (LLMs) undergoing continual learning, as retaining performance across diverse domains is important for their general utility. In this paper, we explore model growth, a promising strategy that leverages smaller models to expedite and structure the training of larger ones for mitigating the catastrophic forgetting problem. Although growth-based pretraining, particularly via transformer stacking, has shown promise in accelerating convergence, its impact on forgetting remains under-explored. Therefore, we evaluate whether growth-based models can retain previously learned capabilities more effectively across a sequence of fine-tuning tasks involving domain knowledge, reasoning, reading comprehension, and bias. Our findings show that both models -- one trained with growth (Stack LLM) and one without (LLM) -- exhibit improvements in domain knowledge. However, reasoning and reading comprehension degrade over time, indicating signs of catastrophic forgetting. Stack LLM consistently shows less degradation, especially in reading comprehension, suggesting enhanced retention capabilities. Interestingly, in bias evaluation, the baseline LLM becomes progressively more neutral with continued fine-tuning, while Stack LLM maintains a steady bias ratio around 60--61\%. These results indicate that growth-based pretraining may deliver modest improvements in resisting catastrophic forgetting, though trade-offs remain in handling social biases.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
