我们研究在计算受限条件下如何高效推理。我们探讨如何在不增加测试时间成本的情况下做出结构化且正确的决策。我们为小型和中型Transformer添加了两个仅在训练时使用的组件,这些组件也适用于更广泛的可微优化器。首先,通过模糊模式位置对齐构建的长度感知注意力先验(RPA)产生一个归一化的softmax前偏置,该偏置在不引入新推理参数的情况下引导注意力,如同一个结构化正则化器。其次,一个最小增益感知控制器Guardian仅在验证改进值得时调整注意力的锐度,遵循非凸优化的双时间尺度策略梯度观点。在推理时,该控制器被禁用。从KL视角来看,softmax of z plus log pi作为MAP在KL正则化下得到支撑,将先验建立在原理性目标上。在WikiText 2上严格计算平衡条件下,我们减少了验证交叉熵,同时保持基线的延迟和内存。在推理时,我们添加一个预计算、缓存的先验B of T作为每个头部的单个加法偏置。控制器不运行。实际上,这带来的开销极小,仅是每个头部的缓存偏置加法,没有可测量的p50延迟变化。我们的结果表明,长度感知先验和后期阶段增益控制在保持测试时间成本不变的同时,保留了稀缺的改进,特别是在长跨度、噪声logit区域。
英文摘要:
We study efficient reasoning under tight compute. We ask how to make structured, correct decisions without increasing test time cost. We add two training only components to small and medium Transformers that also transfer to broader differentiable optimizers. First, a length aware attention prior built via fuzzy regime position alignment, RPA, yields a normalized pre softmax bias that guides attention like a structured regularizer while adding no new inference parameters. Second, a minimal gain aware controller, Guardian, nudges attention sharpness only when validation improvements warrant it, following a two timescale policy gradient view of nonconvex optimization. It is disabled at inference. A KL perspective shows softmax of z plus log pi as MAP with KL regularization, grounding the prior in a principled objective. Under strict compute parity on WikiText 2, we reduce validation cross entropy while matching baseline latency and memory. At inference, we add a precomputed, cached prior B of T as a single additive bias per head. The controller does not run. In practice, this incurs negligible overhead, a cached bias add per head, with no measurable p50 latency shift. Our results suggest that length aware priors and late phase gain control preserve scarce improvements, especially in long span, noisy logit regimes, while keeping test time costs effectively unchanged.
Structured Role-Aware Policy Optimization for Multimodal Reasoning
School of Computing & Data Science, The University of Hong Kong(计算与数据科学学院,香港大学);School of Computing & Data Science and Institute of Data Science, The University of Hong Kong(计算与数据科学学院和数据科学研究所,香港大学)
Mitigating Overthinking in Large Reasoning Models via Difficulty-aware Reinforcement Learning
Laboratory for Artificial Intelligence and New Forms of Education(人工智能与新教育形式实验室);National Engineering Research Center of Educational Big Data(教育大数据国家工程研究中心);Faculty of Artificial Intelligence in Education, Central China Normal University(教育人工智能学院,中央财经大学);College of Electronics and Information Engineering, South-Central Minzu University(电子与信息工程学院,西南民族大学)
BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens
Institute for AI Industry Research (AIR) Tsinghua University(人工智能产业研究院(AIR)清华大学);Global Innovation Exchange & Department of Automation Tsinghua University(全球创新交流中心及自动化系 清华大学)
When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making
Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所);Northeastern University(东北大学);Harvard University(哈佛大学);Cornell University(康奈尔大学);MIT(麻省理工学院);Fujitsu Research of America(美国富士通研究);Tsinghua University(清华大学);Peking University(北京大学);University of Georgia(佐治亚大学);Florida International University(佛罗里达国际大学);EmbodyX Inc(EmbodyX公司);Cisco Systems(思科系统)
FERA: Uncertainty-Aware Federated Reasoning for Large Language Models
Indiana University(印第安纳大学);The University of New South Wales(新南威尔士大学);The Chinese University of Hong Kong(香港中文大学);University of California San Diego(加州大学圣地亚哥分校);Adobe Research(Adobe研究院)
BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts
DISI, University of Trento(特伦托大学DISI学院);DI, University of Pisa(比萨大学DI学院);University of Edinburgh(爱丁堡大学);CIMeC&DISI, University of Trento(特伦托大学CIMeC&DISI学院)
BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMs
The Conversational AI (CoAI) group, DCST, Tsinghua University(清华大学智能对话AI小组,清华大学数据科学与技术研究院);Centre for Frontier AI Research, Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore(前沿人工智能研究中心,高性能计算研究所,新加坡科技研究局);The College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)