SCOPE-OPSD：用于在线策略自蒸馏的Fisher条件特权子空间
SCOPE-OPSD: Fisher-Conditioned Privileged Subspaces for On-Policy Self-Distillation
浏览论文内容
中文总结 AI 辅助
SCOPE-OPSD通过将特权教师-学生残差投影到Fisher条件秩64子空间，为在线策略自蒸馏增加第二监督通道，在Qwen3模型上多数检查点超越纯OPSD和随机对照，验证了数据相关方向的有效性。
中文摘要 AI 辅助
在线策略自蒸馏（OPSD）使用解条件自教师对学生生成的前缀进行评分，但仅通过下一个词元概率传递监督。我们探究对齐的最终层差异是否能提供有用的第二通道，以及如何在不将其几何结构与辅助强度混淆的情况下测试该通道。SCOPE-OPSD将特权教师-学生残差投影到从残差协方差和语言模型头Fisher敏感性估计的冻结秩64因子上。它复用了OPSD已需要的前向传播，不增加回滚或推理时模块。匹配的随机对照保留了结构化因子的秩和非零谱，并使用逐臂梯度均方根校准，隔离了数据相关方向的影响。在Qwen3-1.7B、4B和8B的完整25/50/75/100步轨迹中，结构化方法从未低于纯OPSD，在12个模型-检查点组合中有11个严格提升，在4B第25步完全持平。结构化方法还在12个组合中的10个超过匹配的随机对照。在Qwen3-1.7B的第75步，结构化方法在两个独立的训练重跑中均超过匹配随机对照1.39个Macro Avg@12点。交叉拟合诊断还显示，其保留的特权差距捕获量是匹配随机方向的4.40倍。结果支持在短预算OPSD中使用紧凑的Fisher条件特权子空间。
英文摘要
On-policy self-distillation (OPSD) scores student-generated prefixes with a solution-conditioned self-teacher, yet transfers supervision only through next-token probabilities. We ask whether the aligned final-layer discrepancy offers a useful second channel, and how to test that channel without confusing its geometry with auxiliary strength. SCOPE-OPSD projects the privileged teacher-student residual onto a frozen rank-64 factor estimated from residual covariance and language-model-head Fisher sensitivity. It reuses the forwards already required by OPSD and adds neither rollouts nor inference-time modules. A matched Random control preserves the structured factor's rank and nonzero spectrum and uses per-arm gradient-RMS calibration, isolating the effect of the data-dependent orientation. Across the complete 25/50/75/100-step trajectories for Qwen3-1.7B, 4B, and 8B, Structured is never below Pure OPSD, with strict gains in 11 of the 12 model-checkpoint combinations and an exact tie at 4B step 25. Structured also exceeds matched Random in 10 of the 12 combinations. At step 75 on Qwen3-1.7B, Structured exceeds matched Random by 1.39 Macro Avg@12 points in each of two independent training reruns. A cross-fitted diagnostic also shows 4.40 times greater held-out privileged-gap capture than the matched random orientation. The results support a compact, Fisher-conditioned privileged subspace for short-budget OPSD.
发表机构
- Chongqing Ant Consumer Finance Co., Ltd.（重庆蚂蚁消费金融有限公司）
- Alibaba Cloud Computing Co., Ltd.（阿里云计算有限公司）
- Chongqing University of Posts and Telecommunications（重庆邮电大学）
机构由 AI 辅助整理，请以论文原文为准。
