Reasoning is critical for large language models (LLMs) to excel in a wide range of tasks. While methods like Chain-of-Thought (CoT) reasoning and enhance LLM performance by decomposing problems into intermediate steps, they also incur significant overhead in token usage, leading to increased costs. We find that the reasoning process of current LLMs is unnecessarily lengthy and it can be compressed by including a reasonable token budget in the prompt, but the choice of token budget plays a crucial role in the actual compression effectiveness. We then propose a token-budget-aware LLM reasoning framework that dynamically adjusts the number of reasoning tokens based on the reasoning complexity of each problem. Experiments show that our method effectively reduces token costs in CoT reasoning with only a slight performance reduction, offering a practical solution to balance efficiency and accuracy in LLM reasoning. Code: https://github.com/GeniusHTX/TALE
BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens
Institute for AI Industry Research (AIR) Tsinghua University(人工智能产业研究院(AIR)清华大学);Global Innovation Exchange & Department of Automation Tsinghua University(全球创新交流中心及自动化系 清华大学)
Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models
Department of Computer Science and Engineering, Visvesvaraya Technological University, Belagavi(维斯瓦拉亚科技大学计算机科学与工程系,贝拉加维);Department of Computer Science and Business System, SG Balekundri Institute of Technology, Belagavi(SG巴莱昆德里理工学院计算机科学与商业系统系,贝拉加维)
GMTS: Gradient Magnitude-based Token Selection Improves RLVR Training for LLM Reasoning
School of Mathematical Sciences, Shanghai Jiao Tong University(上海交通大学数学科学学院);Institute of Natural Sciences, Shanghai Jiao Tong University(上海交通大学自然科学研究院)
OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
Zhejiang University(浙江大学);Carnegie Mellon University(卡内基梅隆大学);University of Chinese Academy of Sciences(中国科学院大学);University of Science and Technology of China(中国科学技术大学);Alibaba(阿里巴巴)
On the Token Value Inequality in Efficient Reasoning
Purdue University(普渡大学);Rochester Institute of Technology(罗切斯特理工学院);MIT-IBM Watson AI Lab(MIT-IBM沃森人工智能实验室);University of Missouri-Kansas City(密苏里大学堪萨斯城分校);Rutgers University(罗格斯大学);Meta AI
Not All Tokens Matter: Towards Efficient LLM Reasoning via Token Significance in Reinforcement Learning
Tsinghua University(清华大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Peking University(北京大学);Qwen Large Model Application Team, Alibaba(阿里云文大模型应用团队);Microsoft(微软);Shenzhen Key Laboratory of Ubiquitous Data Enabling, Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院 ubiquitous 数据赋能重点实验室)
SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoning
State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学);BandAI, Bytedance(字节跳动BandAI)
Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学北京校区人工智能学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University College London, University of London(伦敦大学大学学院);Dalian University of Technology(大连理工大学)
In-Context Learning on a Budget: A Case Study in Token Classification
School of Computer Science and Engineering, The Hebrew University of Jerusalem(耶路撒冷希伯来大学计算机科学与工程学院);School of Computing and Information Systems, University of Melbourne(墨尔本大学计算与信息系统学院);Microsoft Corporation(微软公司)
TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs
State Key Lab of CAD&CG, Zhejiang University(浙江大学CAD&CG国家重点实验室);School of Software Technology, Zhejiang University(浙江大学软件学院);Alibaba Cloud(阿里云);Zhiyuan Research Institute(智源研究院)
UHR-BAT: Budget-Aware Token Compression Vision-Language model for Ultra-High-Resolution Remote Sensing
School of Artificial Intelligence Science and Technology, Nanjing University(南京大学人工智能科学与技术学院);School of Electronic Science and Engineering, Nanjing University(南京大学电子科学与工程学院);School of Physics, Nanjing University(南京大学物理学院)
Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Models
University of California, Berkeley(加州大学伯克利分校);University of Cambridge(剑桥大学);University of Washington(华盛顿大学);University of Toronto(多伦多大学);University of Tokyo(东京大学)