arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.13892cs.CL
收藏

越难越好:用更少但更难的数据保持监督微调的泛化能力

The Harder The Better: Maintaining Supervised Fine-tuning Generalization with Less but Harder Data

Beijing Wenge Technology Co.,Ltd(北京文格科技有限公司);College of Intelligence and Computing(智能与计算学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)

Zhaoyang Shang, Sibo Wei, Jianbin Guo, Rui Zhou, Lifeng Dong, Yin Luo

AI总结:

提出THTB框架,通过认知难度评分选择更难的数据,使仅用5%或2%的数据训练即可超越全量训练,提升监督微调的泛化与领域适应能力。

摘要:

大型语言模型(LLMs)在通用任务中表现出色,但将其适应到专业领域依赖于高质量的监督微调(SFT)数据。尽管现有方法能够识别高质量数据的子集并在一定程度上降低训练成本,但其选择过程仍存在过度依赖LLMs内部知识、可解释性弱和泛化能力有限的问题。为解决这些局限,我们提出THTB(越难越好),一个受认知科学启发的指令数据选择与标注指导框架。THTB通过结合质量过滤与内在和外在难度评分,优先考虑更高层次的认知指令,为高效SFT提供可解释且可量化的标准,既适用于数据选择,也适用于标注指导。实验表明,THTB使仅用5%数据训练的模型能够超越全数据集训练,同时相比仅用LLM选择的方法实现更优的泛化。此外,THTB在垂直领域提供有效的标注指导,使仅用2%数据训练的模型能够超越在更大数据集上训练的模型,展现出强大的领域适应潜力。我们的代码、数据集和模型可在https://github.com/DYJG-research/THTB获取。

英文摘要:

Large Language Models (LLMs) excel in general tasks, but adapting them to specialized domains relies on high-quality supervised fine-tuning (SFT) data. Although existing methods can identify subsets of high-quality data and reduce training cost to some extent, their selection process still suffers from over-reliance on LLMs' internal knowledge, weak interpretability, and limited generalization. To address these limitations, we propose THTB (The Harder The Better), a cognitive science-inspired framework for instruction data selection and annotation guidance. THTB prioritizes higher-level cognitive instructions by combining quality filtering with intrinsic and extrinsic hardness scoring, offering interpretable and quantifiable criteria for efficient SFT, both in data selection and annotation guidance. Experiments show that THTB enables models trained on only 5% of the data to outperform full-dataset training, while achieving superior generalization compared with LLM-only selection. In addition, THTB provides effective annotation guidance in vertical domains, enabling a model trained on just 2% of the data to surpass models trained on much larger datasets, demonstrating strong potential for domain adaptation. Our code, datasets, and models are available on https://github.com/DYJG-research/THTB.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑