ReviewInstruct：一种面向大语言模型的评审驱动的多轮对话生成方法
ReviewInstruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language Models
AI总结：
本文提出Review-Instruct框架，通过候选者、多个评审者和主席的迭代“提问-回答-评审”过程生成高质量多轮对话数据，微调LLaMA2-13B后在MMLU-Pro和MT-Bench上分别提升2.9%和2%，验证了评审驱动多智能体框架的有效性。
摘要：
大语言模型（LLMs）在对话式人工智能中的有效性受到其对单轮监督微调（SFT）数据依赖的制约，这限制了多轮对话中的上下文连贯性。现有的多轮对话数据生成方法难以同时保证指令的多样性和质量。为解决这一问题，我们提出了Review-Instruct，这是一个新颖的框架，通过迭代的“提问-回答-评审”过程合成多轮对话，涉及三种智能体角色：一个候选者（Candidate）、多个评审者（Reviewers）和一个主席（Chairman）。该框架通过纳入评审者反馈来迭代优化指令，从而增强对话的多样性和难度。我们使用Alpaca数据集构建了一个多轮数据集，并对LLaMA2-13B模型进行微调。在MT-Bench、MMLU-Pro和Auto-Arena上的评估表明性能显著提升，与基于LLaMA2-13B的先前最先进模型相比，在MMLU-Pro上取得了2.9%的绝对提升，在MT-Bench上取得了2%的绝对提升。消融研究证实了评审阶段以及使用多个评审者在提升指令多样性和难度方面的关键作用。我们的工作凸显了评审驱动的多智能体框架在大规模生成高质量对话数据方面的潜力。
英文摘要：
The effectiveness of large language models (LLMs) in conversational AI is hindered by their reliance on single-turn supervised fine-tuning (SFT) data, which limits contextual coherence in multi-turn dialogues. Existing methods for generating multi-turn dialogue data struggle to ensure both diversity and quality in instructions. To address this, we propose Review-Instruct, a novel framework that synthesizes multi-turn conversations through an iterative "Ask-Respond-Review" process involving three agent roles: a Candidate, multiple Reviewers, and a Chairman. The framework iteratively refines instructions by incorporating Reviewer feedback, enhancing dialogue diversity and difficulty. We construct a multi-turn dataset using the Alpaca dataset and fine-tune the LLaMA2-13B model. Evaluations on MT-Bench, MMLU-Pro, and Auto-Arena demonstrate significant improvements, achieving absolute gains of 2.9\% on MMLU-Pro and 2\% on MT-Bench compared to prior state-of-the-art models based on LLaMA2-13B. Ablation studies confirm the critical role of the Review stage and the use of multiple Reviewers in boosting instruction diversity and difficulty. Our work highlights the potential of review-driven, multi-agent frameworks for generating high-quality conversational data at scale.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
