取其精华,去其糟粕:对大语言模型微调数据选择的再思考
Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Models
AI总结:
本文聚焦大语言模型微调的数据选择,提出特征提取、标准设计、选择器评估的三阶段分类框架及统一比较方法,发现高针对性质量衡量提升效率但降低可行性,并指出未来挑战。
摘要:
大语言模型(LLM)微调中的数据选择旨在从现有数据集中挑选高质量子集,使训练出的模型性能优于在完整数据集上训练的基线模型。然而,不断扩大的研究领域缺乏清晰统一的框架,实验设置的差异性也使得系统性比较变得复杂。尽管现有综述全面概述了数据选择的阶段和方法,但往往忽略了对微调阶段的深入探索。本文对近期用于LLM微调的数据选择技术进行了聚焦性回顾,分析了十余项关键研究。我们提出了一种新颖的三阶段方案——包括特征提取、标准设计和选择器评估——以系统性地分类和评估这些方法。此外,我们提出了一种统一的比较方法,结合基于比率的效率和基于排名的可行性指标,以解决实验中的不一致性。我们的研究结果表明,强调更针对性质量衡量的方法实现了更高的效率,但以牺牲可行性为代价。最后,我们讨论了趋势并强调了微调数据选择中的四个关键挑战,为未来研究提供了潜在方向。
英文摘要:
Data selection for fine-tuning large language models (LLMs) aims to choose a high-quality subset from existing datasets, allowing the trained model to outperform baselines trained on the full dataset. However, the expanding body of research lacks a clear, unified framework, and the variability in experimental settings complicates systematic comparisons. While existing surveys comprehensively overview the stages and methods of data selection, they often overlook an in-depth exploration of the fine-tuning phase. In this paper, we conduct a focused review of recent data selection techniques for fine-tuning LLMs, analyzing a dozen key studies. We introduce a novel three-stage scheme - comprising feature extraction, criteria design, and selector evaluation - to systematically categorize and evaluate these methods. Additionally, we propose a unified comparison approach that incorporates ratio-based efficiency and ranking-based feasibility metrics to address inconsistencies across experiments. Our findings reveal that methods emphasizing more targeted quality measurement achieve higher efficiency but at the cost of feasibility. Finally, we discuss trends and highlight four key challenges in fine-tuning data selection, offering potential directions for future research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。