TRACE：大型语言模型持续学习的综合基准
TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
AI总结：
本文提出TRACE基准，含8个挑战性数据集评估LLM持续学习，发现训练后模型能力显著下降，并引入推理增强持续学习（RCL）方法以减少灾难性遗忘。
摘要：
对齐的大型语言模型（LLMs）在任务解决、指令遵循和安全性方面表现出卓越的能力。然而，这些对齐的LLMs的持续学习方面在很大程度上被忽视了。现有的持续学习基准由于简单性以及模型在指令调优期间可能接触到的数据，对领先的对齐LLMs缺乏足够的挑战性。在本文中，我们引入了TRACE，一个旨在评估LLMs中持续学习的新型基准。TRACE包含8个不同的数据集，涵盖具有挑战性的任务，包括领域特定任务、多语言能力、代码生成和数学推理。所有数据集都标准化为统一格式，允许对LLMs进行轻松自动评估。我们的实验表明，在TRACE上训练后，对齐的LLMs在一般能力和指令遵循能力方面表现出显著下降。例如，llama2-chat 13B在gsm8k数据集上的准确率在训练我们的数据集后从28.8%急剧下降到2%。这突显了在实现特定任务性能的同时保持LLMs原始能力之间找到合适权衡的挑战。实证发现表明，固有配备推理路径的任务有助于显著保持LLMs的某些能力，防止潜在下降。受此启发，我们引入了推理增强持续学习（RCL）方法。RCL将任务特定线索与元理由相结合，有效减少LLMs中的灾难性遗忘，同时加速在新任务上的收敛。
英文摘要：
Aligned large language models (LLMs) demonstrate exceptional capabilities in task-solving, following instructions, and ensuring safety. However, the continual learning aspect of these aligned LLMs has been largely overlooked. Existing continual learning benchmarks lack sufficient challenge for leading aligned LLMs, owing to both their simplicity and the models' potential exposure during instruction tuning. In this paper, we introduce TRACE, a novel benchmark designed to evaluate continual learning in LLMs. TRACE consists of 8 distinct datasets spanning challenging tasks including domain-specific tasks, multilingual capabilities, code generation, and mathematical reasoning. All datasets are standardized into a unified format, allowing for effortless automatic evaluation of LLMs. Our experiments show that after training on TRACE, aligned LLMs exhibit significant declines in both general ability and instruction-following capabilities. For example, the accuracy of llama2-chat 13B on gsm8k dataset declined precipitously from 28.8\% to 2\% after training on our datasets. This highlights the challenge of finding a suitable tradeoff between achieving performance on specific tasks while preserving the original prowess of LLMs. Empirical findings suggest that tasks inherently equipped with reasoning paths contribute significantly to preserving certain capabilities of LLMs against potential declines. Motivated by this, we introduce the Reasoning-augmented Continual Learning (RCL) approach. RCL integrates task-specific cues with meta-rationales, effectively reducing catastrophic forgetting in LLMs while expediting convergence on novel tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
