数据重复胜过数据扩展在长链推理监督微调中
Data Repetition Beats Data Scaling in Long-CoT Supervised Fine-Tuning
AI总结：
本研究发现，在长链推理监督微调中，数据重复训练比数据扩展更有效，通过token准确率作为停止标准可替代昂贵的数据扩展。
摘要：
监督微调（SFT）在链式推理数据上的训练是推理语言模型训练后的关键步骤。标准机器学习直觉认为，使用更多的独特训练样本会获得更好的泛化能力。反直觉的是，我们证明SFT受益于重复：在固定的更新预算下，对较小的数据集进行更多轮次的训练优于对更大数据集进行单轮次的训练。在AIME'24/25和GPQA基准测试中，Olmo3-7B在400个样本上训练128轮次，比在51200个样本上训练1轮次的等效方法在12-26个百分点上表现更优，且没有额外的灾难性遗忘。我们发现，训练token准确率可以可靠地信号重复已饱和；额外轮次的改进在完全记忆化后趋于平稳，这一模式在所有设置中都是一致的。这些发现为推理SFT提供了一种实用方法，其中使用token准确率作为停止标准来扩展轮次可以替代昂贵的无方向数据扩展。我们将重复优势，即完全记忆化与改进的泛化能力相吻合，作为社区理解大语言模型训练动态的新开放问题。
英文摘要：
Supervised fine-tuning (SFT) on chain-of-thought data is an essential post-training step for reasoning language models. Standard machine learning intuition suggests that training with more unique training samples yields better generalization. Counterintuitively, we show that SFT benefits from repetition: under a fixed update budget, training for more epochs on smaller datasets outperforms single-epoch training on larger datasets. On AIME'24/25 and GPQA benchmarks, Olmo3-7B trained for 128 epochs on 400 samples outperforms the equivalent 1 epoch on 51200 samples by 12-26 percentage points, with no additional catastrophic forgetting. We find that training token accuracy reliably signals when repetition has saturated; improvements from additional epochs plateau at full memorization, a pattern consistent across all settings. These findings provide a practical approach for reasoning SFT, where scaling epochs with token accuracy as a stopping criterion can replace expensive undirected data scaling. We pose the repetition advantage, where full memorization coincides with improved generalization, as a new open problem for the community in understanding the training dynamics of large language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
