重新思考大语言模型训练中的数据整理：在线重加权优于离线方法
Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods
AI总结：
本文提出ADAPT框架，通过动态在线重加权提升模型泛化能力，实验显示其在指令微调和大规模预训练中优于传统离线方法。
摘要：
数据整理是大语言模型训练中的关键但研究不足的领域。现有方法如数据选择和混合均采用离线范式，与训练过程脱节，导致工程开销大且整理过程脆弱：整个流程必须在模型/任务变化时重新运行。此外，离线方法通过硬过滤或重采样改变数据量，常牺牲数据多样性并损害泛化能力。本文提出将数据整理重新视为在线重加权问题，通过损失加权动态调整样本重要性而非静态预处理。具体而言，我们引入ADAPT（自适应数据重加权用于预训练和微调），一种动态在线框架，通过基于相似性的质量信号引导的自适应样本学习率重新加权训练样本，不改变训练样本数量。不同于强制静态数据分布的离线方法，ADAPT作为隐式课程学习者，逐步从粗粒度模式转向细粒度语义区分。在指令微调和大规模预训练实验中，ADAPT在相等FLOPs下实现了更强的跨基准泛化能力。
英文摘要：
Data curation is a critical yet under-explored area in large language model (LLM) training. Existing methods, such as data selection and mixing, operate in an offline paradigm, detaching themselves from training. This separation introduces engineering overhead and makes the curation brittle: the entire pipeline must be re-run under model/task shifts. Moreover, offline methods alter data size through hard filtering or resampling, often sacrificing data diversity and harming generalization. We propose to rethink data curation as an online reweighting problem, where sample importance is dynamically adjusted during training via loss weighting rather than static pre-processing. Specifically, we introduce ADAPT (Adaptive Data reweighting for Pretraining and FineTuning), a dynamic online framework that reweights training samples with adaptive per-sample learning rates guided by similarity-based quality signals, without changing the number of training samples. Unlike offline methods that enforce a static data distribution, ADAPT acts as an implicit curriculum learner, progressively shifting focus from coarse-grained patterns to fine-grained semantic distinctions as the model evolves. Experiments on both instruction tuning and large-scale pretraining show that ADAPT consistently outperforms offline selection/mixing and prior online methods, achieving stronger cross-benchmark generalization under equal FLOPs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
