TANDEM: 基于孪生网络的双层数据混合优化
TANDEM: Bi-Level Data Mixture Optimization with Twin Networks
AI总结：
提出TANDEM方法，通过孪生网络（代理模型和参考模型）的差异衡量数据效用，优化领域混合比例，在数据受限和监督微调等场景中显著提升大语言模型性能。
摘要：
大语言模型（LLM）的能力在很大程度上取决于来自不同领域的训练数据。优化特定领域的混合比例可以建模为一个双层优化问题，我们将其简化为单层惩罚形式，并通过孪生网络求解：一个在主数据上训练的代理模型和一个在额外数据上动态更新的参考模型。我们提出的方法——用于双层数据混合优化的孪生网络（TANDEM），通过孪生模型之间的差异衡量数据效用，并增加从额外数据中受益更多的领域的权重。与先前方法相比，TANDEM提供了理论保证和更广泛的适用性。此外，我们的双层视角提出了研究领域重新加权的新设置，例如数据受限场景和监督微调，其中优化的混合比例显著提升了性能。大量实验验证了TANDEM在所有场景中的有效性。
英文摘要：
The capabilities of large language models (LLMs) significantly depend on training data drawn from various domains. Optimizing domain-specific mixture ratios can be modeled as a bi-level optimization problem, which we simplify into a single-level penalized form and solve with twin networks: a proxy model trained on primary data and a dynamically updated reference model trained with additional data. Our proposed method, Twin Networks for bi-level DatA mixturE optiMization (TANDEM), measures the data efficacy through the difference between the twin models and up-weights domains that benefit more from the additional data. TANDEM provides theoretical guarantees and wider applicability, compared to prior approaches. Furthermore, our bi-level perspective suggests new settings to study domain reweighting such as data-restricted scenarios and supervised fine-tuning, where optimized mixture ratios significantly improve the performance. Extensive experiments validate TANDEM's effectiveness in all scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
