RECOST：外部知识引导的数据高效指令微调
RECOST: External Knowledge Guided Data-efficient Instruction Tuning
AI总结：
本文提出RECOST框架，利用外部知识评估LLM合成指令数据，结合重排序与多样性采样，在Alpaca等数据集上仅用1%数据即取得更优效果。
摘要：
在当前大语言模型（LLMs）的发展格局中，指令微调过程是一个关键步骤。考虑到高昂的算力开销，研究者提出了数据高效指令微调，以在这一过程中减少训练数据规模，旨在挑选高质量的指令数据。然而，我们认为当前大多数数据高效指令微调方法高度依赖原始指令微调数据集的质量。当面对由LLMs合成的数据集时——这是该领域常见的场景——脏样本甚至会比其它样本以更高的概率被选中。为应对这些挑战，我们利用外部知识（相关示例或段落），通过基于上下文的相对预测熵来评估那些由LLMs合成的样本。基于这一新度量，我们提出了一个名为RECOST的框架，它将外部知识库重排序和多样性一致采样整合到单一流程中。通过在多个合成数据集（Alpaca和Alpaca-gpt4）上的大量实验，我们证明了该方法的有效性，并且仅使用完整数据集的1%就取得了更好的结果。
英文摘要：
In the current landscape of large language models (LLMs), the process of instruction tuning serves as an essential step. Considering the high computing power overhead, data-efficient instruction tuning was proposed to reduce the training data size in this process, aiming at selecting high-quality instructional data. Nevertheless, we argue that most current data-efficient instruction-tuning methods are highly dependent on the quality of the original instruction-tuning dataset. When it comes to datasets synthesized by LLMs, a common scenario in this field, dirty samples will even be selected with a higher probability than other samples. To address these challenges, we utilized external knowledge (relevant examples or paragraphs) to evaluate those samples synthesized by LLMs with an in-context-based relative predictive entropy. Based on the new metric, we proposed a framework, dubbed as \textbf{RECOST}, which integrates external-knowledge-base re-ranking and diversity-consistent sampling into a single pipeline. Through extensive experiments on several synthetic datasets (Alpaca and Alpaca-gpt4), we demonstrate the effectiveness of our method and achieve even better results with only \textbf{1\%} of the full dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
