arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.19759cs.CLcs.LG
收藏

平衡大语言模型合成数据生成策略的成本与有效性

Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMs

Scale AI

Yung-Chieh Chan, George Pu, Apaar Shanker, Parth Suresh, Penn Jenks, John Heyer, Sam Denton

AI总结:

该研究将LLM合成数据生成策略分为三类,探究不同种子指令集规模和查询预算约束下的策略效果差异,发现最优策略取决于教师查询预算与种子集规模的比值,并提出了适配不同场景的增强方法选择框架。

摘要:

随着大语言模型(LLM)被应用于更多场景,创建高质量、面向特定任务的微调数据集已成为模型提升的瓶颈。使用高质量人工数据一直是释放模型性能的最常用方法,但在许多场景中成本高得令人望而却步。目前也出现了几种替代方法,例如生成合成数据或混合数据,但这些方法的有效性仍不明确,尤其是在资源受限场景和难以验证的任务中。\n为探究这一问题,我们将各类合成数据生成策略划分为三个代表性类别——答案增强(Answer Augmentation)、问题复述(Question Rephrase)和新问题生成(New Question),并研究在种子指令集规模和查询预算两种约束下训练的学生LLM的性能。我们证明这些策略在不同设置下的效果并不相同。值得注意的是,最优数据生成策略很大程度上取决于可用的教师查询预算与种子指令集规模的比值。当该比值较低时,为现有问题生成新答案的效果最佳,但随着比值升高,生成新问题的方式会成为最优选择。\n在所有任务中我们发现,与高数据量 regime 相比,增强方法的选择和其他设计选择在中低数据量 regime 下的影响要大得多。我们提供了一个适用于不同设置的实用框架,用于选择合适的增强方法,该框架还纳入了各方法的可扩展性、合成数据验证的重要性以及使用不同LLM生成合成数据等额外考量因素。

英文摘要:

As large language models (LLMs) are applied to more use cases, creating high quality, task-specific datasets for fine-tuning becomes a bottleneck for model improvement. Using high quality human data has been the most common approach to unlock model performance, but is prohibitively expensive in many scenarios. Several alternative methods have also emerged, such as generating synthetic or hybrid data, but the effectiveness of these approaches remain unclear, especially in resource-constrained scenarios and tasks that are not easily verified. To investigate this, we group various synthetic data generation strategies into three representative categories -- Answer Augmentation, Question Rephrase and New Question -- and study the performance of student LLMs trained under various constraints, namely seed instruction set size and query budget. We demonstrate that these strategies are not equally effective across settings. Notably, the optimal data generation strategy depends strongly on the ratio between the available teacher query budget and the size of the seed instruction set. When this ratio is low, generating new answers to existing questions proves most effective, but as this ratio increases, generating new questions becomes optimal. Across all tasks, we find that choice of augmentation method and other design choices matter substantially more in low to mid data regimes than in high data regimes. We provide a practical framework for selecting the appropriate augmentation method across settings, taking into account additional factors such as the scalability of each method, the importance of verifying synthetic data, and the use of different LLMs for synthetic data generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑