arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.13285cs.AIcs.CL
收藏

BoundaryEvo:面向长链推理蒸馏的能力边界处数据-模型协同进化

BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation

Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Nanjing University(南京大学);Tongji University(同济大学);The Chinese University of Hong Kong(香港中文大学)

Jiaming Su, Suorong Yang, Xudong Li, Tengchao Yang, Peng Ye

AI总结:

本文提出BoundaryEvo框架,通过两阶段数据-模型协同进化,仅用1000个训练示例,在11个推理基准上提升小型语言模型推理能力,效果优于现有基线。

摘要:

从大型教师模型中蒸馏长链推理轨迹是提升小型语言模型推理能力的有效方法。然而,现有方法通常在蒸馏前选择固定的训练集,忽略了推理示例的有用性会随学生模型的进化而变化。当仅能使用少量教师轨迹进行训练时,这种不匹配的代价尤为高昂。此外,已被掌握的示例几乎无法提供新的监督信号,而远超学生能力的示例则可能难以学习。因此,训练数据应随模型能力的进化而调整,但在学生模型初始能力较弱时,可靠的数据选择难以实现。为应对这些挑战,本文的核心思路是先建立基础推理能力,再根据学生模型不断变化的能力边界调整训练数据。我们提出了BoundaryEvo,这是一个两阶段的数据-模型协同进化框架。在能力预训练阶段,我们直接蒸馏少量由教师生成的长推理轨迹,不使用单独的模型对其进行判断和过滤,使学生模型在感知能力的数据选择前获得初始推理能力。在能力边界协同进化阶段,我们反复选择当前学生模型未掌握但专家模型可可靠解决的问题,并随学生模型的进化更新训练数据。仅使用1000个训练示例,BoundaryEvo在11个推理基准上始终优于静态和动态数据选择基线,将原始Qwen3.5-4B在IMO-AnswerBench上的准确率从62.9提升至75.3,在AMO-Bench上的准确率从26.8提升至46.0。代码和数据将公开可用。

英文摘要:

Distilling long-chain reasoning trajectories from large teacher models provides an effective way to improve the reasoning ability of small language models. However, existing methods typically select a fixed training set before distillation, overlooking that the usefulness of a reasoning example changes as the student evolves. This mismatch becomes especially costly when only a small number of teacher trajectories can be used for training. Besides, examples already mastered provide little new supervision, while examples far beyond the capability of student may be difficult to learn from. Training data should therefore adapt as model capability evolves, yet reliable data selection is difficult when the student is initially weak. To address these challenges, our key idea is to first establish the basic reasoning capability and then adapt the training data to the evolving capability boundary of the student. We propose BoundaryEvo, a two-stage data--model co-evolution framework. In Capability Priming, we directly distill a small set of teacher-generated long reasoning trajectories without using a separate model to judge and filter them, allowing the student to acquire an initial reasoning capability before capability-aware data selection. In Capability-Boundary Co-Evolution, we repeatedly select questions that the current student has not mastered but the expert can reliably solve, and update the training data as the student evolves. With only 1,000 training examples, BoundaryEvo consistently outperforms static and dynamic data selection baselines across 11 reasoning benchmarks, raising the original Qwen3.5-4B accuracy from 62.9 to 75.3 on IMO-AnswerBench and from 26.8 to 46.0 on AMO-Bench. The code and data will be made publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑