优化指令合成：基于树搜索的进化空间有效探索
Optimizing Instruction Synthesis: Effective Exploration of Evolutionary Space with Tree Search
AI总结：
本文提出通用可扩展框架IDEA-MCTS，利用蒙特卡洛树搜索和评估模型高效引导指令进化为高质量形态，显著提升数据质量并增强LLMs在低资源下的指令遵循能力。
摘要：
指令微调是将语言模型与人类在现实世界中的实际目标对齐的关键技术。大量研究强调，指令数据的质量对这种对齐的成功至关重要。然而，人工创建高质量数据既费时又费力，这促使研究人员探索使用LLMs来合成数据。近期的研究集中在使用更强的LLM迭代增强现有指令数据，展现出可喜的成果。然而，以往的工作往往缺乏对进化方向的控制，导致数据合成过程存在高不确定性并产生低质量指令。在本文中，我们引入了一个通用且可扩展的框架IDEA-MCTS（使用蒙特卡洛树搜索的指令数据增强），这是一个用于高效合成指令的可扩展框架。借助树搜索和评估模型，它能高效引导每条指令进化为高质量形态，辅助指令微调。实验结果表明，IDEA-MCTS显著增强了种子指令数据，将质量、多样性和复杂性的平均评估分数从2.19提升至3.81。此外，在开放领域基准测试中，实验结果表明IDEA-MCTS在低资源设置下将LLMs现实世界指令遵循技能的准确率平均提高了5%。
英文摘要：
Instruction tuning is a crucial technique for aligning language models with humans' actual goals in the real world. Extensive research has highlighted the quality of instruction data is essential for the success of this alignment. However, creating high-quality data manually is labor-intensive and time-consuming, which leads researchers to explore using LLMs to synthesize data. Recent studies have focused on using a stronger LLM to iteratively enhance existing instruction data, showing promising results. Nevertheless, previous work often lacks control over the evolution direction, resulting in high uncertainty in the data synthesis process and low-quality instructions. In this paper, we introduce a general and scalable framework, IDEA-MCTS (Instruction Data Enhancement using Monte Carlo Tree Search), a scalable framework for efficiently synthesizing instructions. With tree search and evaluation models, it can efficiently guide each instruction to evolve into a high-quality form, aiding in instruction fine-tuning. Experimental results show that IDEA-MCTS significantly enhances the seed instruction data, raising the average evaluation scores of quality, diversity, and complexity from 2.19 to 3.81. Furthermore, in open-domain benchmarks, experimental results show that IDEA-MCTS improves the accuracy of real-world instruction-following skills in LLMs by an average of 5\% in low-resource settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
