探究CoT增强蒸馏的奥秘
Investigating Mysteries of CoT-Augmented Distillation
AI总结：
本文通过消融实验探究CoT增强蒸馏为何有效，发现将CoT序列置于标签之后、使用不连贯理由或仅少量关键token即可取得与完整理由相当的蒸馏提升。
摘要：
引出“思维链”（CoT）理由——即传达“推理”过程的token序列——已被证明能持续提升大语言模型在问答等任务上的表现。更近期的研究表明，这类理由还可用于模型蒸馏：在对小型学生模型进行微调时，除目标标签外再加入从大型“教师”模型引出的CoT序列，往往能带来（通常是显著的）提升。本文提出如下问题：这一额外训练信号为何以及如何有助于模型蒸馏？我们通过消融实验来探究该问题，并报告了一些可能令人意外的结果。具体而言：（1）将CoT序列置于标签之后（而非之前）能持续实现更好的下游性能——这意味着在测试时无需学生进行“推理”即可获得收益。（2）当理由以这种方式追加时，它们不必是连贯的推理序列也能带来提升；例如，性能提升对CoT token的排列具有鲁棒性。事实上，（3）少量关键token就足以实现与在模型蒸馏中使用完整理由时所观察到的等效改进。
英文摘要：
Eliciting "chain of thought" (CoT) rationales -- sequences of token that convey a "reasoning" process -- has been shown to consistently improve LLM performance on tasks like question answering. More recent efforts have shown that such rationales can also be used for model distillation: Including CoT sequences (elicited from a large "teacher" model) in addition to target labels when fine-tuning a small student model yields (often substantial) improvements. In this work we ask: Why and how does this additional training signal help in model distillation? We perform ablations to interrogate this, and report some potentially surprising results. Specifically: (1) Placing CoT sequences after labels (rather than before) realizes consistently better downstream performance -- this means that no student "reasoning" is necessary at test time to realize gains. (2) When rationales are appended in this way, they need not be coherent reasoning sequences to yield improvements; performance increases are robust to permutations of CoT tokens, for example. In fact, (3) a small number of key tokens are sufficient to achieve improvements equivalent to those observed when full rationales are used in model distillation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
