剪枝能否提升推理？基于能力对齐的Long-Cot压缩以获得更好的推理
Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better Reasoning
AI总结：
本文提出Prune-on-Logic框架，通过结构感知剪枝提升Long-CoT推理效率，发现验证剪枝在保持准确性的同时减少令牌使用，揭示剪枝与模型容量对齐的重要性。
摘要：
长链式思考（Long-CoT）推理提高了大型语言模型（LLMs）的准确性，但其冗长、自我反思的风格往往阻碍了有效压缩到小型语言模型（SLMs）的过程。我们通过能力对齐的视角重新审视Long-CoT压缩，并提出问题：剪枝能否提升推理？我们提出了Prune-on-Logic，一个结构感知的框架，将Long-CoT转换为逻辑图，并在自我验证约束下选择性地剪枝低效的推理步骤。通过系统分析三种剪枝策略，针对整个链、核心推理和验证，我们发现验证剪枝在减少令牌使用的同时一致提高准确性，而剪枝推理步骤或随意剪枝会降低性能。我们的研究揭示了有效的剪枝使监督与模型容量对齐，而非仅仅缩短输入。收益在不同任务、模型规模和CoT能力上均成立，较大的模型因推理更丰富但更冗余而受益更多。我们的实证发现强调剪枝作为一种结构优化策略，用于将CoT推理与SLM容量对齐。
英文摘要：
Long chain-of-thought (Long-CoT) reasoning improves accuracy in LLMs, yet its verbose, self-reflective style often hinders effective distillation into small language models (SLMs). We revisit Long-CoT compression through the lens of capability alignment and ask: Can pruning improve reasoning? We propose Prune-on-Logic, a structure-aware framework that transforms Long-CoT into logic graphs and selectively prunes low-utility reasoning steps under self-verification constraints. Through systematic analysis across three pruning strategies targeting entire chains, core reasoning, and verification, we find that verification pruning consistently improves accuracy while reducing token usage, whereas pruning reasoning steps or indiscriminate pruning degrades performance. Our study reveals that effective pruning aligns supervision with model capacity rather than merely shortening inputs. Gains hold across tasks, model scales, and CoT capability, with larger models benefiting more from pruning due to richer but more redundant reasoning. Our empirical findings highlight pruning as a structural optimization strategy for aligning CoT reasoning with SLM capacity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
