SLAP: 基于分层损失剪枝的在线策略数据高效指令微调
SLAP: Stratified Loss-based Pruning for On-Policy Data-Efficient Instruction Tuning
AI总结:
提出SLAP框架,通过分布感知分层采样和相对距离优化实现批次级数据选择,在减少20-40%训练数据的同时保持或提升LLM性能。
摘要:
指令微调优化了大语言模型(LLMs)的专门能力,但通常需要大量数据集和长时间训练。挑战在于通过识别有用数据并高效微调来开发特定能力。高质量且多样化的剪枝数据可以帮助模型以较低成本实现无损性能。在本文中,我们提出 extbf{SLAP},一种新颖的批次感知数据选择框架,评估整个批次组合的可学习性而非单个样本。SLAP通过分布感知分层采样确保全面的数据分布覆盖,同时通过相对距离优化最大化批次内多样性。通过利用Hessian近似的梯度信息进行动态批次选择,SLAP在多种模型架构(LLaMA、ChatGLM)和多样下游任务(包括多轮对话、多语言翻译和问答)上显著优于现有最先进方法。最值得注意的是,与完整数据集训练相比,SLAP在减少20-40%训练数据的情况下实现了更优性能,大幅降低了计算成本,同时保持或提升了模型能力。这些结果确立了SLAP作为大语言模型高效指令微调的有效方法。
英文摘要:
Instruction tuning has optimized the specialized capabilities of large language models (LLMs), but it often requires extensive datasets and prolonged training times. The challenge lies in developing specific capabilities by identifying useful data and efficiently fine-tuning. High-quality and diverse pruned data can help models achieve lossless performance at a lower cost. In this paper, we propose \textbf{SLAP}, a novel batch-aware data selection framework that evaluates the learnability of entire batch compositions rather than individual. SLAP ensures comprehensive data distribution coverage through distribution-aware stratified sampling while maximizing intra-batch diversity through relative distance optimization. By leveraging Hessian-approximated gradient information for dynamic batch selection, SLAP significantly outperforms existing state-of-the-art methods across multiple model architectures (LLaMA, ChatGLM) and diverse downstream tasks including multi-turn dialogue, multilingual translation, and question answering. Most notably, SLAP achieves superior performance with 20-40\% less training data compared to full dataset training, substantially reducing computational costs while maintaining or improving model capabilities. These results establish SLAP as a powerful approach for efficient and effective instruction tuning of large language models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。