arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.23006cs.CL
收藏

InstructDiff:通过微分熵实现领域自适应的数据选择以实现高效的LLM微调

InstructDiff: Domain-Adaptive Data Selection via Differential Entropy for Efficient LLM Fine-Tuning

Peking University(北京大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Tsinghua University(清华大学);SUFE(上海财经大学);SUSTech(南方科技大学)

Junyou Su, He Zhu, Xiao Luo, Liyu Zhang, Hong-Yu Zhou, Yun Chen, Peng Li, Yang Liu, Guanhua Chen

AI总结:

InstructDiff通过微分熵实现领域自适应的数据选择,提升LLM微调效率,实验显示在数学推理和通用指令遵循任务上分别提高17%和52%。

摘要:

监督微调(SFT)是适应大语言模型的基础,但对完整数据集进行训练会带来高昂的成本且边际效益递减。现有数据选择方法存在严重的领域特异性:为通用指令遵循优化的技术在推理任务上失效,反之亦然。我们发现测量基础模型与最小指令微调校准模型之间的熵差会揭示一种模式——具有最低微分熵的样本在不同领域中始终产生最佳性能,但这一原则表现出领域自适应性:推理任务偏好熵增(认知扩展),而通用任务偏好熵减(认知压缩)。我们引入InstructDiff,一种统一框架,通过预热校准、双向NLL过滤和基于熵的排名将微分熵作为领域自适应的选择标准。广泛实验表明,InstructDiff在数学推理上比完整数据训练提高了17%,在通用指令遵循上提高了52%,在使用仅10%的数据的情况下优于先前的基线方法。

英文摘要:

Supervised fine-tuning (SFT) is fundamental to adapting large language models, yet training on complete datasets incurs prohibitive costs with diminishing returns. Existing data selection methods suffer from severe domain specificity: techniques optimized for general instruction-following fail on reasoning tasks, and vice versa. We observe that measuring entropy differences between base models and minimally instruction-tuned calibrated models reveals a pattern -- samples with the lowest differential entropy consistently yield optimal performance across domains, yet this principle manifests domain-adaptively: reasoning tasks favor entropy increase (cognitive expansion), while general tasks favor entropy decrease (cognitive compression). We introduce InstructDiff, a unified framework that operationalizes differential entropy as a domain-adaptive selection criterion through warmup calibration, bi-directional NLL filtering, and entropy-based ranking. Extensive experiments show that InstructDiff achieves 17\% relative improvement over full data training on mathematical reasoning and 52\% for general instruction-following, outperforming prior baselines while using only 10\% of the data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑