通过混合训练缓解数学推理微调中的灾难性遗忘
Mitigating Catastrophic Forgetting in Mathematical Reasoning Finetuning through Mixed Training
AI总结：
本文提出混合训练策略，通过交错数学和NLI任务来缓解微调中的灾难性遗忘，同时保持数学性能和NLI准确性。
摘要：
在为专门任务如数学推理微调大型语言模型时，模型会表现出灾难性遗忘，失去之前学习的能力。我们通过在DeepMind数学数据集上微调Flan-T5-Base（25000万参数）并测量在MultiNLI上的遗忘来研究这一现象。仅数学训练将数学准确性从3.1%提高到12.0%，但导致NLI准确性从81.0%下降到16.5%--在前1000次训练步骤中下降了64.5个百分点。我们提出了混合训练策略，在训练过程中交错数学和NLI示例。我们的结果表明，混合训练完全消除了灾难性遗忘，同时保持等同的数学性能：平衡的1:1比例实现了12.0%的数学准确性（与仅数学训练相同），同时保持86.2%的NLI准确性。我们系统地探索了从1:1到15:1的混合比例，发现即使最小的NLI暴露（6.2%）也能提供有效的正则化。这些发现表明，专业化不需要要求遗忘一般能力，这对扩展到更大模型有影响，其中混合训练可能在防止遗忘之外还带来额外的好处。
英文摘要：
When finetuning large language models for specialized tasks such as mathematical reasoning, models exhibit catastrophic forgetting, losing previously learned capabilities. We investigate this by finetuning Flan-T5-Base (250M parameters) on the DeepMind Mathematics dataset and measuring forgetting on MultiNLI. Math-only training improves mathematical accuracy from 3.1\% to 12.0\% but causes NLI accuracy to collapse from 81.0\% to 16.5\%--a 64.5 percentage point drop occurring within the first 1,000 training steps. We propose mixed training strategies that interleave mathematical and NLI examples during training. Our results demonstrate that mixed training completely eliminates catastrophic forgetting while maintaining equivalent mathematical performance: the balanced 1:1 ratio achieves 12.0\% math accuracy (matching math-only) while preserving 86.2\% NLI accuracy. We systematically explore mixing ratios from 1:1 to 15:1, finding that even minimal NLI exposure (6.2\%) provides effective regularization. These findings demonstrate that specialization need not require forgetting general capabilities, with implications for scaling to larger models where mixed training may confer additional benefits beyond forgetting prevention.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
