Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs), yet it incurs substantial computational overhead for inference. Existing CoT compression methods often suffer from a critical loss of logical fidelity at high compression ratios, resulting in significant performance degradation. To achieve high-fidelity, fast reasoning, we propose a novel EXTreme-RAtio Chain-of-Thought Compression framework, termed Extra-CoT, which aggressively reduces the token budget while preserving answer accuracy. To generate reliable, high-fidelity supervision, we first train a dedicated semantically-preserved compressor on mathematical CoT data with fine-grained annotations. An LLM is then fine-tuned on these compressed pairs via a mixed-ratio supervised fine-tuning (SFT), teaching it to follow a spectrum of compression budgets and providing a stable initialization for reinforcement learning (RL). We further propose Constrained and Hierarchical Ratio Policy Optimization (CHRPO) to explicitly incentivize question-solving ability under lower budgets by a hierarchical reward. Experiments on three mathematical reasoning benchmarks show the superiority of Extra-CoT. For example, on MATH-500 using Qwen3-1.7B, Extra-CoT achieves over 73\% token reduction with an accuracy improvement of 0.6\%, significantly outperforming state-of-the-art (SOTA) methods. Our source codes have been released at https://github.com/Mwie1024/Extra-CoT.
Reinforcement Learning for Chain of Thought Compression with One-Domain-to-All Generalization
CFCS, School of Computer Science, Peking University(计算机学院,北京大学);State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学);LLM-Core Xiaomi(小米LLM-Core)
GraphComp: Extreme Error-bounded Compression of Scientific Data via Temporal Graph Autoencoders
Computer, Electrical and Mathematical Sciences and Engineering Division, KAUST(卡奥斯特大学计算机、电气与数学科学与工程分校);Department of Informatics and Telecommunications, University of Peloponnese(希腊皮埃尼亚斯大学信息与电信系);Physical Science and Engineering Division, KAUST(卡奥斯特大学物理科学与工程分校)
Training with Quantization Noise for Extreme Model Compression
Facebook AI Research(Facebook人工智能研究院);LORIA(洛里亚计算机科学实验室);Univ Lyon(里昂大学);Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);ENS de Lyon(里昂高等师范学校);UCB Lyon 1(里昂第一大学);LIP UMR 5668(里昂第一大学计算机科学实验室)
The Spectral Geometry of Thought: Phase Transitions, Instruction Reversal, Token-Level Dynamics, and Perfect Correctness Prediction in How Transformers Reason
Crystal-KV: Efficient KV Cache Management for Chain-of-Thought LLMs via Answer-First Principle
School of Computer Science and Technology, University of Science and Technology of China(计算机科学与技术学院,中国科学技术大学);University of Science and Technology of China(中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(人工智能研究所,合肥综合性国家科学中心);Suzhou Institute for Advanced Research, University of Science and Technology of China(苏州先进研究院,中国科学技术大学)
CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks
Multiverse Computing(多维计算公司);Catalan Institute of Nanoscience and Nanotechnology (ICN2)(加泰罗尼亚纳米科学与纳米技术研究所);The Barcelona Institute of Science and Technology(巴塞罗那科学技术研究院);Donostia International Physics Center(多斯蒂亚国际物理中心);Centre for Social Innovation(社会创新中心)