更短的思考，同样的答案：基于难度的分段强化学习用于CoT压缩
SCA: Segment-Wise CoT Compression with Answer Alignment
AI总结：
本文提出DSS-GRPO方法，通过分段强化学习实现CoT推理的压缩，以在保持答案准确性的同时减少推理长度。
摘要：
链式思维（CoT）提高了推理的可靠性，但增加了token成本，这促使对显式推理轨迹进行后训练压缩。然而，最短的充分推理并不普遍：它取决于难度、模型容量和训练状态，使固定长度目标变得脆弱。在实践中，基于朴素RL的压缩也可能不希望地缩短用户面对的答案，因为单个完成级的学习信号会跨越思考/答案边界。我们提出了难度缩放的分段GRPO（DSS-GRPO），它将回报分解为思考和答案组件，计算每个分段的组相对优势，并通过硬token掩码路由它们，使压缩更新仅作用于思考，而答案对齐仅作用于答案。DSS-GRPO使用提示内的组内塑造和难度感知缩放来鼓励简洁的推理，而不会导致答案行为崩溃。
英文摘要：
Chain-of-thought (CoT) reasoning improves problem solving, but long think traces increase inference cost. Existing CoT compression methods usually optimize completion-level length. For structured thinking models, however, a completion contains both a think segment and an answer segment, so completion-level compression can save tokens by compressing not only the CoT but also the answer. We call this failure mode answer drift. We propose Segment-wise CoT Compression with Answer Alignment (SCA), an answer-preserving think-compression method. SCA parses completions into functional segments, routes compression rewards only to successful think tokens, and protects answer tokens through length and distribution alignment to a frozen base model. Experiments show that, across datasets from multiple domains, SCA achieves state-of-the-art-level chain-of-thought compression while preserving the base model's performance and answer alignment. Training data and code are included in the supplementary code and data package.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
