超滤：用于快速指令微调的弱到强数据过滤
Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
AI总结：
本文提出“超滤”方法，利用更小更弱的模型筛选指令数据以微调更大更强的模型，在降低过滤成本的同时提升微调效率和模型性能。
摘要：
指令微调对于提升大语言模型（LLMs）至关重要，但通常受到低质量和冗余数据的困扰。针对指令微调的数据过滤已被证明对提高微调过程的效率和性能十分重要，但由于该过程涉及大语言模型，也会带来额外的成本和计算开销。为了降低过滤成本，我们研究了“超滤”（Superfiltering）问题：能否使用一个更小、更弱的模型来筛选数据，以微调一个更大、更强的模型？尽管弱语言模型与强语言模型之间存在性能差距，我们发现它们在感知指令难度和数据筛选结果方面具有高度一致的能力。这使我们能够使用一个小得多且更高效的模型来过滤用于训练更大语言模型的指令数据。这不仅大幅加快了数据过滤速度，而且经过过滤数据微调的大语言模型在标准基准上甚至取得了更好的性能。大量实验验证了我们方法的有效性和高效性。
英文摘要：
Instruction tuning is critical to improve LLMs but usually suffers from low-quality and redundant data. Data filtering for instruction tuning has proved important in improving both the efficiency and performance of the tuning process. But it also leads to extra cost and computation due to the involvement of LLMs in this process. To reduce the filtering cost, we study Superfiltering: Can we use a smaller and weaker model to select data for finetuning a larger and stronger model? Despite the performance gap between weak and strong language models, we find their highly consistent capability to perceive instruction difficulty and data selection results. This enables us to use a much smaller and more efficient model to filter the instruction data used to train a larger language model. Not only does it largely speed up the data filtering, but the filtered-data-finetuned LLM achieves even better performance on standard benchmarks. Extensive experiments validate the efficacy and efficiency of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
