arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.08659cs.CL
收藏

CODA:面向自适应推理的难度感知计算分配

CODA: Difficulty-Aware Compute Allocation for Adaptive Reasoning

Fudan University(复旦大学)

Siye Wu, Jian Xie, Yikai Zhang, Yanghua Xiao

AI总结:

本文提出CODA,通过难度感知信号动态分配计算资源,实现自适应推理。在易任务中降低token成本,在难任务中提升性能。

摘要:

大推理模型的出现表明,增加推理时间的计算显著提升了复杂任务的性能。然而,它常陷入另一个陷阱:过度思考简单问题,重复的推理理由带来最小的准确性提升,而成本却不成比例地高。这促使自适应推理:动态地将推理深度与实例难度对齐。本文从最优性角度研究自适应推理,将其形式化为一个效用最大化问题,其中token的分配持续到边际准确性增益低于增量成本。基于此,我们提出了CODA(通过难度感知的计算分配),一种通过策略内部难度信号分配token的方法。具体而言,CODA通过基于组的回滚来估计难度,并将其映射到两个非负门控,调节一个长度依赖的塑造项,以二进制基础奖励之上。易侧门控对简单实例惩罚冗长性,而难侧门控鼓励在具有挑战性的情况下进行更多反思性回滚。在模型规模和基准测试中,CODA在没有外部注释或用户提供的预算的情况下实现了自适应推理:在易任务中,CODA将token成本降低了超过60%,同时保持强大的准确性,而在难任务中,它激励更多反思性回滚以最大化性能。

英文摘要:

The emergence of large reasoning models demonstrates that scaling inference-time compute significantly enhances performance on complex tasks. However, it often falls into another trap: overthinking simple problems, where repetitive rationales yield minimal accuracy gains at a disproportionately high cost. This motivates adaptive reasoning: dynamically aligning reasoning depth with instance difficulty. In this paper, we study adaptive reasoning from an optimality perspective, formalizing it as a utility maximization problem where tokens are allocated until the marginal accuracy gain falls below the incremental cost. Based on this, we propose CODA (Compute Allocation by Difficulty Awareness), a method that operationalizes this principle by allocating tokens via a policy-internal difficulty signal. Specifically, CODA estimates difficulty via group-based rollouts and maps it to two non-negative gates that modulate a length-dependent shaping term on top of the binary base reward. The easy-side gate penalizes verbosity on simple instances, whereas the hard-side gate encourages more deliberative rollouts on challenging ones. Across model scales and benchmarks, CODA achieves adaptive reasoning without external annotations or user-provided budgets: on easy tasks, CODA reduces token costs by over 60% while maintaining strong accuracy, whereas on hard tasks it incentivizes more deliberative rollouts to maximize performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑