arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.29957cs.SEcs.LG
收藏

在代码生成中任意思考

Think Anywhere in Code Generation

School of Computer Science, Peking University(1 计算机科学系,北京大学);Tongyi Lab, Alibaba Group(2 龙洋实验室,阿里巴巴集团)

Xue Jiang, Tianyu Zhang, Ge Li, Mengyang Liu, Taozhi Chen, Zhenhua Xu, Binhua Li, Wenpin Jiao, Zhi Jin, Yongbin Li, Yihong Dong

AI总结:

本文提出Think-Anywhere机制,使LLM在代码生成过程中可按需调用推理,通过冷启动训练和基于结果的强化学习奖励,实现适应性推理,提升代码生成性能和可解释性。

摘要:

近年来,基于大型语言模型(LLMs)的推理进展主要依赖于前置思考,即在最终答案前进行推理。然而,这种方法在代码生成中存在关键限制,因为问题的完整复杂性通常在代码实现过程中才显现。此外,它无法在代码生成过程中自适应地分配推理努力,因为难度差异显著。本文提出Think-Anywhere,一种新的推理机制,使LLM能够在代码生成过程中任意位置调用思考。我们通过首先教授LLM模仿推理模式进行冷启动训练,然后利用基于结果的强化学习奖励来驱动模型自主探索何时和在哪里调用推理。在四个主流代码生成基准(即LeetCode、LiveCodeBench、HumanEval和MBPP)上的大量实验表明,Think-Anywhere在现有推理方法和最近的后训练方法上均取得最佳性能,同时在不同LLM上表现出一致的泛化能力。我们的分析进一步揭示,Think-Anywhere使模型能够自适应地在高熵位置调用推理,提供增强的可解释性。

英文摘要:

Recent advances in reasoning Large Language Models (LLMs) have primarily relied on upfront thinking, where reasoning occurs before final answer. However, this approach suffers from critical limitations in code generation, where upfront thinking is often insufficient as problems' full complexity only reveals itself during code implementation. Moreover, it cannot adaptively allocate reasoning effort throughout the code generation process where difficulty varies significantly. In this paper, we propose Think-Anywhere, a novel reasoning mechanism that enables LLMs to invoke thinking on-demand at any token position during code generation. We achieve Think-Anywhere by first teaching LLMs to imitate the reasoning patterns through cold-start training, then leveraging outcome-based RL rewards to drive the model's autonomous exploration of when and where to invoke reasoning. Extensive experiments on four mainstream code generation benchmarks (i.e., LeetCode, LiveCodeBench, HumanEval, and MBPP) show that Think-Anywhere achieves state-of-the-art performance over both existing reasoning methods and recent post-training approaches, while demonstrating consistent generalization across diverse LLMs. Our analysis further reveals that Think-Anywhere enables the model to adaptively invoke reasoning at high-entropy positions, providing enhanced interpretability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑