CEM:一种数据高效的方法,使大语言模型从错误中持续进化
CEM: A Data-Efficient Method for Large Language Models to Continue Evolving From Mistakes
AI总结:
提出CEM方法,通过迭代评估和补充错误相关知识来收集数据,结合持续指令微调与持续预训练,显著提升大语言模型在问答任务上的性能,最高提升29.63%。
摘要:
随着世界知识的进步和新任务模式的出现,持续学习(CL)对于保持大语言模型(LLM)的时效性和解决其不足变得至关重要。该过程通常涉及持续指令微调(CIT)和持续预训练(CPT),以使这些模型能够适应新任务并获取关键知识。然而,收集足够的CPT数据并高效地弥合知识差距仍然是重大挑战。受“总结错误”策略的启发,我们提出了从错误中持续进化(CEM)方法,这是一种数据高效的方法,旨在通过迭代评估和补充与错误相关的知识来收集CPT数据并持续提升LLM的性能。为了进一步优化数据使用并缓解遗忘,我们引入了一种结合CIT和CPT的新型训练范式。实验表明,CEM显著提升了多个模型在领域内和领域外问答任务上的性能,最高提升了29.63%。代码和数据集可在https://anonymous.4open.science/r/cem-BB25获取。
英文摘要:
As world knowledge advances and new task schemas emerge, Continual Learning (CL) becomes essential for keeping Large Language Models (LLMs) current and addressing their shortcomings. This process typically involves continual instruction tuning (CIT) and continual pre-training (CPT) to enable these models to adapt to novel tasks and acquire critical knowledge. However, collecting sufficient CPT data and efficiently bridging knowledge gaps remain significant challenges. Inspired by the 'summarizing mistakes' strategy, we propose the Continue Evolving from Mistakes (CEM) method, a data-efficient approach aiming to collect CPT data and continually improve LLMs' performance through iterative evaluation and supplementation with mistake-relevant knowledge. To further optimize data usage and mitigate forgetting, we introduce a novel training paradigm that combines CIT and CPT. Experiments show that CEM substantially enhances multiple models' performance on both in-domain and out-of-domain QA tasks, achieving gains of up to 29.63%. Code and datasets are available on https://anonymous.4open.science/r/cem-BB25.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。