选择性反思调优：学生选择的数据回收用于大语言模型指令调优
Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
AI总结：
本文提出选择性反思调优，通过教师LLM反思改进数据质量、学生LLM选择兼容样本，自动精炼现有指令数据，在Alpaca和WizardLM上训练出顶尖7B和13B模型。
摘要：
指令调优对于大语言模型（LLMs）实现更好的指令遵循和任务适应能力至关重要，但其成功在很大程度上依赖于训练数据质量。许多近期方法专注于提升数据质量，却常常忽视数据与被微调的学生模型之间的兼容性。本文提出选择性反思调优（Selective Reflection-Tuning），这是一种新颖的范式，将教师LLM的反思与内省能力用于改进现有数据质量，并与学生LLM的数据选择能力协同，自动精炼现有指令调优数据。这种师生协作能够生成高质量且与学生兼容的指令-响应对，从而实现样本高效的指令调优和性能更优的LLMs。选择性反思调优是一种数据增强与合成方法，无需收集全新数据即可普遍改善LLM微调和自我改进。我们将该方法应用于Alpaca和WizardLM数据，获得了性能更强且达到顶尖水平的7B和13B LLMs。
英文摘要：
Instruction tuning is critical to large language models (LLMs) for achieving better instruction following and task adaptation capabilities but its success heavily relies on the training data quality. Many recent methods focus on improving the data quality but often overlook the compatibility of the data with the student model being finetuned. This paper introduces Selective Reflection-Tuning, a novel paradigm that synergizes a teacher LLM's reflection and introspection for improving existing data quality with the data selection capability of the student LLM, to automatically refine existing instruction-tuning data. This teacher-student collaboration produces high-quality and student-compatible instruction-response pairs, resulting in sample-efficient instruction tuning and LLMs of superior performance. Selective Reflection-Tuning is a data augmentation and synthesis that generally improves LLM finetuning and self-improvement without collecting brand-new data. We apply our method to Alpaca and WizardLM data and achieve much stronger and top-tier 7B and 13B LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
