Star-Agents：基于LLM智能体的指令微调数据自动优化
Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning
AI总结：
Star-Agents框架通过多LLM智能体协作生成、双模型评估和动态精炼自动优化指令微调数据质量，实验表明优化后数据集平均提升12%，并在Fermi等指标上最高提升40%。
摘要：
大语言模型（LLMs）在下游任务上的效果通常取决于指令微调，而指令微调又关键依赖于训练数据的质量。然而，收集高质量且多样化的数据既昂贵又耗时。为缓解这一问题，我们提出了一种新颖的Star-Agents框架，该框架通过多智能体协作与评估来自动提升各数据集的数据质量。该框架采用三管齐下的策略：首先，通过定制的采样方法，利用多个LLM智能体生成多样化的指令数据；随后，生成的数据经过严格评估，采用双模型方法同时评估难度和质量；最后，上述过程在动态精炼阶段中不断演进，优先使用更有效的LLM，从而提升整体数据质量。我们的实证研究（包括使用Pythora和LLaMA等模型进行的指令微调实验）证明了所提框架的有效性。优化后的数据集取得了显著提升，平均提高12%，并在特定指标上获得显著增益，例如在Fermi上提升40%，这些结果得到了MT-bench、Vicuna bench和WizardLM testset等基准的验证。
英文摘要：
The efficacy of large language models (LLMs) on downstream tasks usually hinges on instruction tuning, which relies critically on the quality of training data. Unfortunately, collecting high-quality and diverse data is both expensive and time-consuming. To mitigate this issue, we propose a novel Star-Agents framework, which automates the enhancement of data quality across datasets through multi-agent collaboration and assessment. The framework adopts a three-pronged strategy. It initially generates diverse instruction data with multiple LLM agents through a bespoke sampling method. Subsequently, the generated data undergo a rigorous evaluation using a dual-model method that assesses both difficulty and quality. Finaly, the above process evolves in a dynamic refinement phase, where more effective LLMs are prioritized, enhancing the overall data quality. Our empirical studies, including instruction tuning experiments with models such as Pythia and LLaMA, demonstrate the effectiveness of the proposed framework. Optimized datasets have achieved substantial improvements, with an average increase of 12% and notable gains in specific metrics, such as a 40% improvement in Fermi, as evidenced by benchmarks like MT-bench, Vicuna bench, and WizardLM testset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
