Supervised fine-tuning (SFT) is fundamental to adapting large language models, yet training on complete datasets incurs prohibitive costs with diminishing returns. Existing data selection methods suffer from severe domain specificity: techniques optimized for general instruction-following fail on reasoning tasks, and vice versa. We observe that measuring entropy differences between base models and minimally instruction-tuned calibrated models reveals a pattern -- samples with the lowest differential entropy consistently yield optimal performance across domains, yet this principle manifests domain-adaptively: reasoning tasks favor entropy increase (cognitive expansion), while general tasks favor entropy decrease (cognitive compression). We introduce InstructDiff, a unified framework that operationalizes differential entropy as a domain-adaptive selection criterion through warmup calibration, bi-directional NLL filtering, and entropy-based ranking. Extensive experiments show that InstructDiff achieves 17\% relative improvement over full data training on mathematical reasoning and 52\% for general instruction-following, outperforming prior baselines while using only 10\% of the data.
School of Computer and Communication Engineering, University of Science and Technology Beijing(计算机与通信工程学院,科学技术大学);National Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence(通用人工智能国家重点实验室,北京通用人工智能研究院);Shunde Innovation School, University of Science and Technology Beijing(顺德创新学院,科学技术大学)
LP-SFT: Keeping Plausible Alternatives Alive in Supervised Fine-Tuning
School of Mathematical Sciences, Peking University(北京大学数学科学学院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);College of Computing, Georgia Institute of Technology(佐治亚理工学院计算学院)
SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Meituan(美团);Shanghai Jiao Tong University(上海交通大学)
Shanghai Jiao Tong University(上海交通大学);Shanghai Innovation Institute(上海创新研究院);Large Language Model Department, Tencent(腾讯大语言模型部);University of Macau(澳门大学)
KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier
School of Computer Science, Fudan University(复旦大学计算机科学技术学院);Shanghai Key Laboratory of Intelligent Information Processing, Fudan University(复旦大学上海市智能信息处理重点实验室)
预训练模型被广泛用于微调下游任务,线性分类器通过交叉熵损失进行优化,这可能面临鲁棒性和稳定性问题。这些问题可以通过学习表示来改善,这些表示在进行预测时关注同一类中的相似性和不同类中的矛盾。在本文中,我们利用K-最近邻分类器进行预训练模型的微调。对于这个KNN分类器,我们引入了一个有监督的动量对比学习框架来学习有监督的下游任务的聚类表示。大量的文本分类任务和鲁棒性测试的实验表明,通过将KNN与传统的微调过程中,我们可以获得显着的提高清洁的准确性在丰富的源和Few-Shot设置,可以提高对对抗攻击的鲁棒性。\footnote{all codes is available at https://github.com/LinyangLee/KNN-BERT}
Distance Metric Learning Loss Functions in Few-Shot Scenarios of Supervised Language Models Fine-Tuning
Warsaw University of Technology(华沙理工大学);NASK - National Research Institute(NASK国家研究所)
本文分析了距离度量学习(DML)损失函数对分类任务的语言模型的监督微调的影响。我们使用来自SentEval Transfer Tasks的已知数据集进行了实验。我们的实验表明,应用DML损失函数可以提高Few-Shot场景中RoberTa大型模型的下游分类任务的性能。使用SoftTriple损失进行微调的模型可以比使用标准分类交叉熵损失函数的模型获得更好的结果,从0.04到13.48个百分点,具体取决于训练数据集。此外,我们完成了一个全面的分析与解释性技术,以评估模型的可靠性和解释他们的结果。
EntropyLong: Effective Long-Context Training via Predictive Uncertainty
School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);Tsinghua University(清华大学)