The emergence of large reasoning models demonstrates that scaling inference-time compute significantly enhances performance on complex tasks. However, it often falls into another trap: overthinking simple problems, where repetitive rationales yield minimal accuracy gains at a disproportionately high cost. This motivates adaptive reasoning: dynamically aligning reasoning depth with instance difficulty. In this paper, we study adaptive reasoning from an optimality perspective, formalizing it as a utility maximization problem where tokens are allocated until the marginal accuracy gain falls below the incremental cost. Based on this, we propose CODA (Compute Allocation by Difficulty Awareness), a method that operationalizes this principle by allocating tokens via a policy-internal difficulty signal. Specifically, CODA estimates difficulty via group-based rollouts and maps it to two non-negative gates that modulate a length-dependent shaping term on top of the binary base reward. The easy-side gate penalizes verbosity on simple instances, whereas the hard-side gate encourages more deliberative rollouts on challenging ones. Across model scales and benchmarks, CODA achieves adaptive reasoning without external annotations or user-provided budgets: on easy tasks, CODA reduces token costs by over 60% while maintaining strong accuracy, whereas on hard tasks it incentivizes more deliberative rollouts to maximize performance.
Probing the Difficulty Perception Mechanism of Large Language Models
Institute of Digital Twin, EIT(数字孪生研究所,EIT);Wuhan University of Science and Technology(武汉科技大学);Zhejiang University(浙江大学);Hong Kong Polytechnic University(香港理工大学);Shenzhen Institutes of Advanced Technology, CAS(深圳先进技术研究院,中国科学院)
Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Department of Computer Science, National University of Singapore(新加坡国立大学计算机科学系);Ant Group(蚂蚁集团)
LyriCAR: A Difficulty-Aware Curriculum Reinforcement Learning Framework For Controllable Lyric Translation
School of Informatics, Xiamen University(厦门大学信息学院);School of Journalism and Communication, Xiamen University(厦门大学新闻传播学院);School of Film, Xiamen University(厦门大学电影学院);Xiamen Key Laboratory of Intelligent Storage and Computing, Xiamen University(厦门大学智能存储与计算重点实验室);Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能研究院)
Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning
State Key Lab of Processors, Institute of Computing Technology, CAS(处理器国家重点实验室,计算技术研究所,中国科学院);University of Science and Technology of China(中国科学技术大学);University of Chinese Academy of Sciences(中国科学院大学);StepFun Inc(StepFun公司)