Large Language Models (LLMs) often generate unnecessarily verbose Chain-of-Thought (CoT) reasoning that increases computational costs and latency without proportional performance gains. In this paper, we propose Fine-grained Group policy Optimization (FGO), a Reinforcement Learning (RL) algorithm that refines group responses by subdividing them and assigning appropriate weights based on length and entropy, thereby enabling effective CoT compression. Meanwhile, as an enhanced variant of Group Relative Policy Optimization (GRPO), FGO successfully addresses two major limitations of the GRPO: inefficient data utilization and entropy collapse. We evaluate FGO on multiple reasoning LLMs and benchmarks, including MATH500, AIME24, AMC23, and Minerva. Experimental results show that FGO achieves efficient CoT compression without degrading performance, and simultaneously resolves the key limitations of GRPO. Code: https://github.com/Mr-XcHan/FGO.
Reinforcement Learning for Chain of Thought Compression with One-Domain-to-All Generalization
CFCS, School of Computer Science, Peking University(计算机学院,北京大学);State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学);LLM-Core Xiaomi(小米LLM-Core)
Chain Of Thought Compression: A Theoretical Analysis
School of Computer and Information Technology, Shanxi University, Taiyuan, Shanxi, China(山西大学计算机与信息学院);Queen Mary, University of London, UK(伦敦大学女王学院);School of Informatics, University of Edinburgh, UK(爱丁堡大学信息学院)
D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学(深圳));TeleAI of China Telecom(中国电信TeleAI);The Hong Kong Polytechnic University(香港理工大学);The University of Hong Kong(香港大学)
Can Reasoning Path still be Effective as Input? Bridging Post-Reasoning to Chain-of-Thought Compression
Faculty of Electronic and Information Engineering, Xi’an Jiaotong University(西安交通大学电子与信息工程学院);Queen Mary University of London(伦敦大学玛丽女王学院);City University of Hong Kong(香港城市大学)
SEER: Self-Enhancing Chain-of-Thought Compression for Reasoning Models
The State Key Laboratory of Blockchain and Data Security, Zhejiang University(区块链与数据安全国家重点实验室,浙江大学);State Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学)
Fre-Res: Frequency-Residual Video Token Compression for Efficient Video MLLMs
College of Computer Science, National University of Defense Technology(计算机科学学院,国防科技大学);Shien-Ming Wu School of Intelligent Engineering, South China University of Technology(智能工程谢民明伍学院,华南理工大学)
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);Department of Computer Science, The Hong Kong Polytechnic University(香港理工大学计算机科学系)
Crystal-KV: Efficient KV Cache Management for Chain-of-Thought LLMs via Answer-First Principle
School of Computer Science and Technology, University of Science and Technology of China(计算机科学与技术学院,中国科学技术大学);University of Science and Technology of China(中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(人工智能研究所,合肥综合性国家科学中心);Suzhou Institute for Advanced Research, University of Science and Technology of China(苏州先进研究院,中国科学技术大学)
CoVisco: Codec-Native Vision Encoder with Native Token Compression for Unified Image-Video Understanding
ERNIE Team, Baidu Inc.(百度公司ERNIE团队);The Hong Kong University of Science and Technology(香港科技大学);Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所)
HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference
School of Software Technology, Zhejiang University(浙江大学软件学院);China Mobile (Zhejiang) Research & Innovation Institute(中国移动(浙江)研究院);State Key Lab of CAD&CG, Zhejiang University(浙江大学CAD与CG国家重点实验室);Geely Automobile Research Institute (Ningbo) Co., Ltd.(吉利汽车研究院(宁波)有限公司);FABU Inc.(FABU公司)
Tight Compression: Compressing CNN Through Fine-Grained Pruning and Weight Permutation for Efficient Implementation
ACCESS - AI Chip Center for Emerging Smart Systems(ACCESS——新兴智能系统人工智能芯片中心);Hong Kong University of Science and Technology (HKUST)(香港科技大学);NVIDIA Corporation(英伟达公司)
Prediction of Compression Index of Fine-Grained Soils Using a Gene Expression Programming Model
Ferdowsi University of Mashhad(马什哈德菲尔多西大学);Khorasan Construction Engineering organization(呼罗珊建筑工程组织);Michael Baker International(迈克尔·贝克国际公司);Oxford Brookes University(牛津布鲁克斯大学);Obuda University(奥布达大学);Hakim Sabzevari University(哈基姆·萨布泽瓦里大学)
Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition
Department of Computer Science, Technical University of Munich(计算机科学系,慕尼黑技术大学);Schulich School of Engineering, University of Calgary(工程学院,卡尔加里大学);Google DeepMind, Canada(加拿大谷歌深Mind)
PCNN: Pattern-based Fine-Grained Regular Pruning towards Optimizing CNN Accelerators
Tsinghua University(清华大学);Northeastern University(东北大学);Xi’an Jiaotong University(西安交通大学);Institute for Interdisciplinary Information Core Technology(交叉信息核心技术研究院)
The Spectral Geometry of Thought: Phase Transitions, Instruction Reversal, Token-Level Dynamics, and Perfect Correctness Prediction in How Transformers Reason