聚类与排序：通过专家对齐的质量估计进行保持多样性的指令选择
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation
AI总结：
本文提出一种工业友好且保持多样性的指令数据选择方法CaR，通过专家对齐的评分模型排序和聚类两步流程，仅用1.96%的Alpaca数据便使模型性能平均提升32.1%，同时大幅降低成本。
摘要：
借助开源社区的贡献，涌现了大量指令微调（IT）数据。鉴于训练和评估模型需要大量的资源分配，拥有一种高效选择高质量IT数据的方法是有益的。然而，现有的指令数据选择方法存在局限性，例如依赖脆弱的外部API、受GPT模型偏差影响，或降低了所选指令数据集的多样性。在本文中，我们提出了一种工业友好、专家对齐且保持多样性的指令数据选择方法：Clustering and Ranking（CaR）。CaR采用两步流程：首先，使用与专家偏好对齐的高准确率（84.25%）评分模型对指令对进行排序；其次，通过聚类保持数据集的多样性。在我们的实验中，CaR仅高效选择了Alpaca的1.96%的IT数据，但在GPT-4评估中，生成的AlpaCaR模型性能平均超过Alpaca 32.1%。此外，我们发现，无论预训练模型能力更强还是模型参数规模扩大，数据选择都是一种一致的范式。我们的方法采用具有550M参数的紧凑模型，仅花费当前方法11.2%的财务支出，从而增强了其工业可部署性。
英文摘要：
With contributions from the open-source community, a vast amount of instruction tuning (IT) data has emerged. Given the significant resource allocation required for training and evaluating models, it is advantageous to have an efficient method for selecting high-quality IT data. However, existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset. In this paper, we propose an industrial-friendly, expert-aligned and diversity-preserved instruction data selection method: Clustering and Ranking (CaR). CaR employs a two-step process: first, it ranks instruction pairs using a high-accuracy (84.25%) scoring model aligned with expert preferences; second, it preserves dataset diversity through clustering. In our experiment, CaR efficiently selected a mere 1.96% of Alpaca's IT data, yet the resulting AlpaCaR model surpassed Alpaca's performance by an average of 32.1% in GPT-4 evaluations. Moreover, we find that data selecting is a consistent paradigm whether the pre-trained model is more capable or the model parameters scaling up. Our approach employs compact models with 550M parameters and incurs just 11.2% of the financial outlay of current methods, enhancing its industrial deployability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
