AVSD:通过平衡共识和教师特定的特权信号实现自适应视图自蒸馏
AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals
AI总结:
本文提出AVSD,一种通过平衡共识和教师特定的特权信号来实现自适应视图自蒸馏的方法,以解决自蒸馏中教师和学生信息不对称和特权信息选择的问题。
摘要:
自蒸馏使语言模型能够通过使用同一模型作为学生和教师来从自身轨迹中学习,其中教师基于学生无法访问的特权信息进行条件。此类信息可以是不同种类或视图,如解决方案、演示、反馈或最终答案。这种设置可以在不依赖外部模型的情况下提供密集的token级反馈,但会产生根本性的不对称性:教师可能依赖于视图特定的信息,而学生在推理时无法访问。此外,最佳的特权信息类型通常是任务依赖的,使得选择单一教师视图变得困难。在本工作中,我们通过引入AVSD(自适应视图自蒸馏),一种具有多种特权信息视图的自蒸馏新方法,来同时解决这两个挑战。AVSD通过分离稳定的跨视图共识和视图特定的残差信号来重建token级监督。AVSD识别出跨视图共享的共识信号,提供可靠的更新方向,然后在两者一致且比例适当的情况下,选择性地添加视图特定的残差信号以调整更新幅度。在数学竞赛基准(AIME24、AIME25和HMMT25)上的实验表明,AVSD在Qwen3-8B和Qwen3-4B上分别比单视图自蒸馏基线和GRPO平均Avg@8提升了3.1%和2.2%。此外,在代码生成基准(Codeforces、LiveCodeBench v6)上使用Qwen3-8B时,AVSD在平均上比单视图自蒸馏基线高出2.4%。
英文摘要:
Self-distillation enables language models to learn on-policy from their own trajectories by using the same model as both student and teacher, with the teacher being conditioned on privileged information unavailable to the student. Such information can come in different types or views, such as solutions, demonstrations, feedback, or final answers. This setup provides dense token-level feedback without relying on a separate external model, but creates a fundamental asymmetry: the teacher may rely on view-specific information that the student cannot access at inference time. Moreover, the best type of privileged information is often task-dependent, making it difficult to choose a single teacher view. In this work, we address both these challenges jointly by introducing AVSD (Adaptive-View Self-Distillation), a novel method of self-distillation with multiple privileged-information views, which reconstructs token-level supervision by separating stable cross-view consensus from view-specific residual signals. AVSD identifies the consensus signal shared across views, which provides a reliable update direction, and then selectively adds the view-specific residual signal to adjust the update magnitude when it both aligns with the consensus direction and remains proportionate to the consensus signal. Experiments on math competition benchmarks (AIME24, AIME25, and HMMT25) show that AVSD consistently outperforms both single-view self-distillation baselines and GRPO, achieving average Avg@8 gains of 3.1% and 2.2% over the strongest baselines on Qwen3-8B and Qwen3-4B, respectively. Moreover, on code-generation benchmarks (Codeforces, LiveCodeBench v6) using Qwen3-8B, AVSD outperforms the single-view self-distillation baseline by 2.4% on average.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。