何时压缩有益，何时有害：链式思维蒸馏的条件感知分析
When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation
AI总结：
通过系统实验揭示链式思维蒸馏中压缩方法的关键因素：重要性准则的粒度、重构级别和压缩预算在不同领域和模式下的影响，并给出条件感知的部署指南。
摘要：
链式思维（CoT）蒸馏将多步推理从大型推理模型转移到较小的学生模型，但冗长的教师轨迹增加了训练和推理成本。现有的CoT压缩方法分为两类：选择性剪枝和生成式重写，但先前的研究混淆了关键因素：剪枝中粒度与重要性准则混淆，重写中重构级别很少被孤立研究，且压缩预算未在领域或模式间系统评估。我们沿三个维度重新审视CoT压缩：重要性准则、重构级别和压缩预算。在两种模型系列（数学和通用领域）以及长/短CoT模式下进行系统扫描，我们发现：（i）重要性准则的效用严格受粒度控制：步骤级准则收敛于共享推理骨干，而令牌级剪枝需要符号感知信号以保留逻辑核心；（ii）重构级别在领域间反转：数学任务随结构破坏单调下降，而激进重写在通用任务中充当去噪器；（iii）训练时压缩不一定转化为推理时节省：长CoT学生尽管有简洁监督仍保留冗长习惯，使得训练比率成为部署成本的一个乐观下界。这些发现为匹配压缩与部署上下文提供了条件感知指南。
英文摘要：
Chain-of-Thought (CoT) distillation transfers multi-step reasoning from large reasoning models to smaller students, but verbose teacher traces inflate both training and inference cost. Existing CoT compression methods fall into two families, selective pruning and generative rewriting, yet prior studies have left key factors entangled: granularity is confounded with importance criteria in pruning, restructuring level is rarely isolated in rewriting, and compression budgets are not systematically evaluated across domains or regimes. We recast CoT compression along three dimensions: importance criterion, restructuring level, and compression budget. Sweeping these across two model families, Math and General domains, and Long-/Short-CoT regimes, we find that (i) importance criterion utility is strictly governed by granularity: step-level criteria converge on a shared reasoning backbone, while token-level pruning requires symbol-aware signals to preserve the logical core; (ii) restructuring level inverts across domains: Math degrades monotonically with structural disruption, while aggressive rewriting acts as a denoiser on General tasks; (iii) training-time compression does not necessarily translate to inference-time savings: Long-CoT students retain verbose habits despite concise supervision, making the training ratio an optimistic lower bound on deployment cost. These findings yield condition-aware guidelines for matching compression to deployment context.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
