Recent advances in reasoning Large Language Models (LLMs) have primarily relied on upfront thinking, where reasoning occurs before final answer. However, this approach suffers from critical limitations in code generation, where upfront thinking is often insufficient as problems' full complexity only reveals itself during code implementation. Moreover, it cannot adaptively allocate reasoning effort throughout the code generation process where difficulty varies significantly. In this paper, we propose Think-Anywhere, a novel reasoning mechanism that enables LLMs to invoke thinking on-demand at any token position during code generation. We achieve Think-Anywhere by first teaching LLMs to imitate the reasoning patterns through cold-start training, then leveraging outcome-based RL rewards to drive the model's autonomous exploration of when and where to invoke reasoning. Extensive experiments on four mainstream code generation benchmarks (i.e., LeetCode, LiveCodeBench, HumanEval, and MBPP) show that Think-Anywhere achieves state-of-the-art performance over both existing reasoning methods and recent post-training approaches, while demonstrating consistent generalization across diverse LLMs. Our analysis further reveals that Think-Anywhere enables the model to adaptively invoke reasoning at high-entropy positions, providing enhanced interpretability.
University of Science and Technology of China(中国科学技术大学);Alibaba Inc.(阿里巴巴);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳大数据研究院)
School of Computing, National University of Singapore, Singapore(新加坡国立大学计算机学院);Department of Statistics, University of Oxford, United Kingdom(英国牛津大学统计系);School of Intelligence Science(智能科学学院);Technology, Nanjing University, China(技术学院,南京大学,中国)
ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of British Columbia(不列颠哥伦比亚大学);University College London(伦敦大学学院)
Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Moonshot AI;Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of Electronic Science and Technology of China(电子科技大学);School of Artifcial Intelligence, Wuhan University(武汉大学人工智能学院);Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MoE(下一代智能搜索与推荐工程技术研究中心)
A Study on Thinking Patterns of Large Reasoning Models in Code Generation
Nanyang Technological University(南洋理工大学);Institute for Infocomm Research (I2R), A*Star(信息通信研究所(I2R),A*Star);Southern Cross University(南十字星大学);National University of Singapore(新加坡国立大学)
Aletheia: What Makes RLVR For Code Verifiers Tick?
INSAIT, Sofia University "St. Kliment Ohridski", Bulgaria(保加利亚索菲亚大学INSAIT实验室);Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt and National Research Center for Applied Cybersecurity(德累斯顿技术大学计算机科学系及应用网络安全国家研究中心通用知识处理实验室);ATHENE, Germany(德国ATHENE研究院)