arXiv 2509.137902025-11-03cs.CLcs.AI

因材施教！基于能力感知课程学习的LLM指令微调

Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning

Tsinghua University（清华大学）；Peng Cheng Laboratory（鹏城实验室）；Cornell University（康奈尔大学）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；Guangming Laboratory（光明实验室）

Yangning Li, Tingwei Lu, Yinghui Li, Yankai Chen, Wei-Chieh Huang, Wenhao Jiang, Hui Wang, Hai-Tao Zheng, Philip S. Yu

备注：: EMNLP 2025 Findings

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对当前课程微调方法因依赖静态难度指标导致课程僵化的问题，提出能力感知多视角课程指令微调框架CAMPUS，通过动态选择子课程、能力感知调整进度及多种难度调度，显著提升了LLM指令微调性能。

摘要：

高效的指令微调旨在提升在给定指令数据集上训练的大型语言模型（LLMs）的最终性能。课程学习作为一种典型的数据组织策略，在指令微调中已展现出初步的有效性。然而，当前的课程微调方法存在课程僵化的问题，因为它们仅依赖静态的启发式难度指标。这些方法无法适应模型在训练过程中不断演化的能力，导致学习轨迹固定且可能处于次优状态。为解决该问题，提出了名为CAMPUS（Competence-Aware Multi-Perspective cUrriculum inStruction tuning framework，能力感知多视角课程指令微调框架）的方法。CAMPUS具有以下优势：（1）子课程的动态选择。（2）对课程进度的能力感知调整。（3）多种基于难度的调度。大量实验证明，与其他用于高效指令微调的最先进基线方法相比，CAMPUS具有卓越的性能。

英文摘要：

Efficient instruction tuning aims to enhance the ultimate performance of large language models (LLMs) trained on a given instruction dataset. Curriculum learning as a typical data organization strategy has shown preliminary effectiveness in instruction tuning. However, current curriculum tuning methods suffer from the curriculum rigidity, since they rely solely on static heuristic difficulty metrics. These methods fail to adapt to the evolving capabilities of models during training, resulting in a fixed and potentially sub-optimal learning trajectory. To address the issue, Competence-Aware Multi-Perspective cUrriculum inStruction tuning framework termed CAMPUS is proposed. CAMPUS offers several advantages: (1) Dynamic selection for sub-curriculum. (2) Competency-aware adjustment to the curriculum schedule. (3) Multiple difficulty-based scheduling. Extensive experiments prove the superior performance of CAMPUS, compared to other state-of-the-art baselines for efficient instruction tuning.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。