最强的教师并不总是最好的教师：以学生为中心的答案选择
When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection
AI总结：
提出以学生为中心的答案采样（SCAS）框架，通过估计学生中心的学习成本选择教师生成的答案，从而提升学生模型性能。
摘要：
LLM训练越来越依赖教师生成的监督，包括合成响应、推理轨迹和工具使用演示。当前实践通常选择表现最好的教师来生成学生训练数据，隐含地将教师测试表现视为教学质量的代理。我们表明这一假设可能失败：即使多个教师对同一问题提供正确答案，最强教师的答案也不一定是对给定学生的最佳监督。为解决这一问题，我们提出以学生为中心的答案采样（SCAS），该框架根据估计的学生中心学习成本从经过验证的教师生成答案中进行选择。受逐词梯度分解的启发，我们推导出该成本的高效前向代理，并在训练中用于指导答案选择。在30个教师模型、6个学生基础模型和8个任务上的实验表明，SCAS持续提升学生性能，表明有效的蒸馏应优先考虑与当前学生匹配的监督，而非仅依赖教师强度。
英文摘要：
LLM training increasingly relies on teacher-generated supervision, from synthetic responses to reasoning traces and tool-use demonstrations. Current practice often chooses the highest-performing teacher to generate student training data, implicitly treating teacher test performance as a proxy for teaching quality. We show that this assumption can fail: even when multiple teachers provide correct answers to the same question, the answer from the strongest teacher is not necessarily the best supervision for a given student. To address this gap, we propose Student-Centric Answer Sampling (SCAS), a framework that selects from verified teacher-generated answers according to their estimated student-centric learning cost. Motivated by a token-wise gradient decomposition, we derive an efficient forward-only proxy for this cost and use it to guide answer selection during training. Experiments across 30 teacher models, 6 student base models, and 6 tasks show that SCAS consistently improves student performance, suggesting that effective distillation should prioritize supervision matched to the current student rather than teacher strength alone.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
