始终学习，始终混合：始终高效且简单的数据混合
Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time
AI总结：
本文提出OP-Mix算法，通过低秩适配器插值实现数据混合，提升预训练和持续学习性能，减少计算资源消耗。
摘要：
数据混合决定了如何结合不同来源或类型的数据，是语言模型训练中的关键问题。在预训练中，数据组合是模型质量的关键决定因素；在持续学习和适应中，它决定了保留和获取的内容。然而，现有数据混合方法只解决生命周期中的一个阶段：一些需要较小的代理模型绑定到单个训练阶段，其他假设固定领域集，持续学习缺乏系统指导。我们认为数据混合本质上是一个在线决策问题——在整个训练过程中反复出现，需要单一统一的解决方案。我们引入OP-Mix（On-Policy Mix）算法，该算法贯穿语言模型训练的整个生命周期。我们的主要见解是候选数据混合可以通过插值低秩适配器来廉价模拟，这些适配器直接在当前模型上训练，消除了单独的代理模型，确保搜索始终基于模型的实际学习动态。在预训练、持续中训练和持续指令微调中，OP-Mix始终找到接近最优的混合，同时使用比基线少的计算资源。在预训练中，OP-Mix在平均困惑度上比不混合训练提高了6.3%。对于持续学习，OP-Mix在使用66%和95%更少计算的情况下，性能与重新训练和在线策略蒸馏相当。OP-Mix表明语言模型训练的不同观点：不是一系列不同的阶段，而是一个连续的数据学习过程。
英文摘要：
Data mixing decides how to combine different sources or types of data and is a consequential problem throughout language model training. In pretraining, data composition is a key determinant of model quality; in continual learning and adaptation, it governs what is retained and acquired. Yet existing data mixing methods address only one phase of this lifecycle at a time: some require smaller proxy models tied to a single training phase, others assume a fixed domain set, and continual learning lacks principled guidance altogether. We argue that data mixing is fundamentally an online decision making problem -- one that recurs throughout training and demands a single, unified solution. We introduce OP-Mix (On-Policy Mix), a data mixing algorithm that operates across the entire language model training lifecycle. Our main insight is that candidate data mixtures can be cheaply simulated by interpolating between low-rank adapters trained directly on the current model, eliminating separate proxy models and ensuring the search is always grounded in the model's actual learning dynamics. Across pretraining, continual midtraining, and continual instruction tuning, OP-Mix consistently finds near-optimal mixtures while using a fraction of the compute of the baselines. In pretraining, OP-Mix improves upon training without mixing by 6.3% in average perplexity. For continual learning, OP-Mix matches the performance of both retraining and on-policy distillation while using 66% and 95% less overall compute, respectively. OP-Mix suggests a different view of language model training: not a sequence of distinct phases, but a single continuous process of learning from data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
