arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.10430cs.CL
收藏

较小的语言模型能够为较大的语言模型选择指令微调训练数据

Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models

University of California San Diego(加利福尼亚大学圣迭戈分校);Halıcıoğlu Data Science Institute, University of California San Diego(加利福尼亚大学圣迭戈分校哈利乔格鲁数据科学学院)

Dheeraj Mekala, Alex Nguyen, Jingbo Shang

AI总结:

本文提出基于样本学习百分比的训练数据选择方法,发现较小语言模型能为较大模型筛选高质量指令微调数据,在降低成本的同时实现相当或更优的性能。

摘要:

指令微调语言模型已成为使其适应通用用途的关键步骤。通常,这一过程涉及在大型数据集上进行广泛训练,产生高昂的训练成本。在本文中,我们引入了一种基于样本学习百分比的全新训练数据选择方法。我们断言,当前的语言模型具备自主选择高质量训练数据的能力,与在整个数据集上训练相比,能带来相当或更优的性能。我们的实验跨越了不同规模的模型,揭示了该特性适用于从1B(小)到13B(大)规模的模型。此外,我们展示了一个有趣的发现:数据难度可以跨模型规模迁移,较小的350M模型能够有效地为较大的13B模型筛选包含困难样本的高质量训练数据,从而产生与在完整数据集上训练相比同等或更优的指令微调模型。利用规模达13B的开源OPT和Llama-2模型、两个公开可用的指令微调训练数据集,并通过自动指标和人工双重评估,本文引入了一种新颖的训练数据选择方法,展示了一种更高效的替代方案。

英文摘要:

Instruction-tuning language models has become a crucial step in aligning them for general use. Typically, this process involves extensive training on large datasets, incurring high training costs. In this paper, we introduce a novel training data selection based on the learning percentage of the samples. We assert that current language models possess the capability to autonomously select high-quality training data, leading to comparable or improved performance compared to training on the entire dataset. Our experiments span different-sized models, revealing that this characteristic holds for models ranging from 1B (small) to 13B (large) in size. Moreover, we demonstrate an interesting finding that the data hardness transfers across model sizes, and a smaller 350M model can effectively curate high-quality training data with hard samples for a larger 13B model, resulting in an equally or superior instruction-tuned model compared to training on the complete dataset. Utilizing open-sourced OPT and Llama-2 models up to 13B in size, two publicly available instruction-tuning training datasets and evaluated by both automatic metrics & humans, our paper introduces a novel approach to training data selection, showcasing a more efficient alternative.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑