Reflection-tuning：数据回收改进LLM指令微调
Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning
AI总结：
提出名为reflection-tuning的新方法，利用预言机LLM回收并提升原始训练数据质量，有效解决低质量数据对指令微调的负面影响并提升模型性能。
摘要：
大型语言模型（LLM）的最新进展扩展了自然语言理解与生成的边界。LLM的输出控制及其与输入的对齐可以通过指令微调来报告。正如先前研究所强调，训练集中的低质量数据通常对指令微调有害，导致LLM输出不一致甚至具有误导性。我们提出了一种新方法，称为“reflection-tuning”，通过LLM的自我改进和自我判断能力来解决这一问题。该方法利用预言机LLM来回收原始训练数据，通过内省来提升指令和响应的质量。在广泛使用的评估基准上进行的广泛实验表明，使用我们回收数据训练的LLM在各种基准上优于使用原始数据集训练的LLM。
英文摘要：
Recent advancements in Large Language Models (LLMs) have expanded the horizons of natural language understanding and generation. Notably, the output control and alignment with the input of LLMs can be refined through instruction tuning. However, as highlighted in several studies, low-quality data in the training set are usually detrimental to instruction tuning, resulting in inconsistent or even misleading LLM outputs. We propose a novel method, termed "reflection-tuning," which addresses the problem by self-improvement and judging capabilities of LLMs. This approach utilizes an oracle LLM to recycle the original training data by introspecting and enhancing the quality of instructions and responses in the data. Extensive experiments on widely used evaluation benchmarks show that LLMs trained with our recycled data outperform those trained with existing datasets in various benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
