arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2602.08324cs.LG
收藏

通过极端比例思维链压缩实现高效大型语言推理模型

Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression

University of Science and Technology of China(中国科学技术大学)

Yuntian Tang, Bohan Jia, Wenxuan Huang, Lianyue Zhang, Jiao Xie, Wenxi Li, Wei Li, Jie Hu, Xinghao Chen Rongrong Ji, Shaohui Lin

AI总结:

提出Extra-CoT框架,通过极端比例压缩思维链、混合比例监督微调和约束层次化比率策略优化,在显著减少推理令牌的同时保持甚至提升推理准确率。

摘要:

思维链推理成功增强了大型语言模型的推理能力,但推理时会产生大量计算开销。现有的思维链压缩方法在高压缩比下常遭受关键逻辑保真度的损失,导致性能显著下降。为实现高保真、快速推理,我们提出了一种新颖的极端比例思维链压缩框架,称为Extra-CoT,该框架在保留答案准确性的同时,激进地减少令牌预算。为了生成可靠的高保真监督,我们首先在带有细粒度标注的数学思维链数据上训练一个专用的语义保留压缩器。然后,通过混合比例监督微调对大型语言模型进行微调,使其学习遵循一系列压缩预算,并为强化学习提供稳定的初始化。我们进一步提出约束和层次化比率策略优化,通过层次化奖励明确激励在较低预算下的问题解决能力。在三个数学推理基准上的实验显示了Extra-CoT的优越性。例如,在MATH-500上使用Qwen3-1.7B,Extra-CoT实现了超过73%的令牌减少,同时准确率提升0.6%,显著优于最先进方法。我们的源代码已在https://github.com/Mwie1024/Extra-CoT发布。

英文摘要:

Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs), yet it incurs substantial computational overhead for inference. Existing CoT compression methods often suffer from a critical loss of logical fidelity at high compression ratios, resulting in significant performance degradation. To achieve high-fidelity, fast reasoning, we propose a novel EXTreme-RAtio Chain-of-Thought Compression framework, termed Extra-CoT, which aggressively reduces the token budget while preserving answer accuracy. To generate reliable, high-fidelity supervision, we first train a dedicated semantically-preserved compressor on mathematical CoT data with fine-grained annotations. An LLM is then fine-tuned on these compressed pairs via a mixed-ratio supervised fine-tuning (SFT), teaching it to follow a spectrum of compression budgets and providing a stable initialization for reinforcement learning (RL). We further propose Constrained and Hierarchical Ratio Policy Optimization (CHRPO) to explicitly incentivize question-solving ability under lower budgets by a hierarchical reward. Experiments on three mathematical reasoning benchmarks show the superiority of Extra-CoT. For example, on MATH-500 using Qwen3-1.7B, Extra-CoT achieves over 73\% token reduction with an accuracy improvement of 0.6\%, significantly outperforming state-of-the-art (SOTA) methods. Our source codes have been released at https://github.com/Mwie1024/Extra-CoT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑