SMART：面向指令微调的子模数据混合策略
SMART: Submodular Data Mixture Strategy for Instruction Tuning
AI总结：
针对指令微调中任务比例平衡难、缺乏系统方法的问题，提出基于子模函数的SMART数据混合策略，可分配任务权重、筛选无冗余样本，在有限预算下性能优于传统混合方法。
摘要：
指令微调指在一组指令格式数据集上对语言模型进行微调，以提升模型对未见过任务的泛化能力。已有研究表明微调过程中平衡不同任务占比十分重要，但找到合适的平衡比例仍具挑战性。遗憾的是，目前除了手动调参或依赖从业者经验直觉外，尚无系统性方法。本文提出SMART（Submodular data Mixture strAtegy for instRuction Tuning，面向指令微调的子模数据混合策略）——一种新型数据混合策略，利用子模函数为各任务分配重要性分数，再据此确定混合权重。在给定微调预算的条件下，SMART会在各任务间重新分配预算，并从每个任务中筛选无冗余的样本。实验结果表明，SMART的性能显著优于示例比例混合、等比例混合等传统方法。此外，SMART支持仅基于少量代表性任务子集构建数据混合方案；通过任务剪枝分析我们发现，在预算有限的场景下，将预算分配给代表性任务子集，相比将预算分配给所有任务能取得更优性能。复现实验结果的代码已开源在https://github.com/kowndinya-renduchintala/SMART。
英文摘要：
Instruction Tuning involves finetuning a language model on a collection of instruction-formatted datasets in order to enhance the generalizability of the model to unseen tasks. Studies have shown the importance of balancing different task proportions during finetuning, but finding the right balance remains challenging. Unfortunately, there's currently no systematic method beyond manual tuning or relying on practitioners' intuition. In this paper, we introduce SMART (Submodular data Mixture strAtegy for instRuction Tuning) - a novel data mixture strategy which makes use of a submodular function to assign importance scores to tasks which are then used to determine the mixture weights. Given a fine-tuning budget, SMART redistributes the budget among tasks and selects non-redundant samples from each task. Experimental results demonstrate that SMART significantly outperforms traditional methods such as examples proportional mixing and equal mixing. Furthermore, SMART facilitates the creation of data mixtures based on a few representative subsets of tasks alone and through task pruning analysis, we reveal that in a limited budget setting, allocating budget among a subset of representative tasks yields superior performance compared to distributing the budget among all tasks. The code for reproducing our results is open-sourced at https://github.com/kowndinya-renduchintala/SMART.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
