arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.15570cs.CLcs.LG
收藏

DEM:用于混合数据分布训练的分布编辑模型

DEM: Distribution Edited Model for Training with Mixed Data Distributions

AGI Foundations, Amazon(亚马逊AGI基金会);AWS AI Labs(亚马逊云科技人工智能实验室)

Dhananjay Ram, Aditya Rawal, Momchil Hardalov, Nikolaos Pappas, Sheng Zha

AI总结:

本文提出分布编辑模型(DEM),通过逐元素向量操作组合各数据源独立训练的模型与基础模型,以低成本高效优化混合数据分布训练,显著提升多基准性能,且修改单数据源无需重新训练。

摘要:

使用混合数据分布进行训练是创建多任务和指令跟随模型的一个常见且重要的部分。数据分布的多样性和联合训练的成本使得优化过程极具挑战性。数据混合方法部分解决了这个问题,尽管在不同数据源上性能次优,并且需要多次昂贵的训练运行。在本文中,我们提出了一种简单高效的替代方案,通过将每个数据源单独训练的模型与基础模型使用基本的逐元素向量操作相结合,来更好地优化数据源。由此产生的模型,即分布编辑模型(DEM),比标准数据混合便宜11倍,并在各种基准测试中优于强基线,在3B到13B规模的模型上,MMLU提升高达6.2%,BBH提升11.5%,DROP提升16.1%,MathQA提升6%,HELM提升9.3%。值得注意的是,当修改单个数据源时,DEM不需要完全重新训练,从而使其在训练多样数据源时非常灵活和可扩展。

英文摘要:

Training with mixed data distributions is a common and important part of creating multi-task and instruction-following models. The diversity of the data distributions and cost of joint training makes the optimization procedure extremely challenging. Data mixing methods partially address this problem, albeit having a sub-optimal performance across data sources and require multiple expensive training runs. In this paper, we propose a simple and efficient alternative for better optimization of the data sources by combining models individually trained on each data source with the base model using basic element-wise vector operations. The resulting model, namely Distribution Edited Model (DEM), is 11x cheaper than standard data mixing and outperforms strong baselines on a variety of benchmarks, yielding upto 6.2% improvement on MMLU, 11.5% on BBH, 16.1% on DROP, 6% on MathQA, and 9.3% on HELM with models of size 3B to 13B. Notably, DEM does not require full re-training when modifying a single data-source, thus making it very flexible and scalable for training with diverse data sources.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑