ConMax：用于高效思维链推理的置信度最大化压缩
ConMax: Confidence-Maximizing Compression for Efficient Chain-of-Thought Reasoning
AI总结：
ConMax通过强化学习框架在保持推理模式的同时，高效压缩推理轨迹，提升大规模推理模型的效率与性能
摘要：
近年来，大型推理模型（LRMs）的突破表明，广泛的思维链（CoT）生成对于启用复杂的认知行为，如自我验证和回溯，以解决复杂任务至关重要。然而，这种能力常常导致"过度思考"，即模型生成冗余的推理路径，这些路径会增加计算成本而不会提高准确性。虽然在推理轨迹上进行监督微调（SFT）是冷启动阶段的标准范式，但将现有压缩技术应用于这些轨迹往往会损害逻辑连贯性或带来压制性的采样成本。在本文中，我们介绍了ConMax（置信度最大化压缩），一种新的强化学习框架，旨在自动压缩推理轨迹，同时保持必要的推理模式。ConMax将压缩制定为奖励驱动的优化问题，通过冻结的辅助LRM，训练策略通过最大化预测保真度的置信度和推理有效性的置信度的加权组合来修剪冗余。在五个推理数据集上进行的广泛实验表明，ConMax实现了优越的效率-性能权衡。具体来说，它在强基线上的推理长度减少了43%，而仅以0.7%的准确性下降为代价，证明了其在为LRMs生成高质量、高效的训练数据方面的有效性。
英文摘要：
Recent breakthroughs in Large Reasoning Models (LRMs) have demonstrated that extensive Chain-of-Thought (CoT) generation is critical for enabling intricate cognitive behaviors, such as self-verification and backtracking, to solve complex tasks. However, this capability often leads to ``overthinking'', where models generate redundant reasoning paths that inflate computational costs without improving accuracy. While Supervised Fine-Tuning (SFT) on reasoning traces is a standard paradigm for the 'cold start' phase, applying existing compression techniques to these traces often compromises logical coherence or incurs prohibitive sampling costs. In this paper, we introduce ConMax (Confidence-Maximizing Compression), a novel reinforcement learning framework designed to automatically compress reasoning traces while preserving essential reasoning patterns. ConMax formulates compression as a reward-driven optimization problem, training a policy to prune redundancy by maximizing a weighted combination of answer confidence for predictive fidelity and thinking confidence for reasoning validity through a frozen auxiliary LRM. Extensive experiments across five reasoning datasets demonstrate that ConMax achieves a superior efficiency-performance trade-off. Specifically, it reduces inference length by 43% over strong baselines at the cost of a mere 0.7% dip in accuracy, proving its effectiveness in generating high-quality, efficient training data for LRMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
