ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference
Nanyang Technological University(南洋理工大学);Carnegie Mellon University(卡内基梅隆大学);Northeast University (Qinhuangdao Campus)(东北大学(秦皇岛校区));Beihang University(北航);University of the Chinese Academy of Sciences(中国科学院大学);Renmin University of China(中国人民大学);Sun Yat-sen University(中山大学)
Siyuan Ma, Bo Gao, Xiaojun Jia, Simeng Qin, Tianlin Li, Ke Ma, Xiaoshuang Jia, Wenqi Ren, Yang Liu
大型语言模型(LLM)推理范式正从参数扩展转向测试时计算扩展,然而许多现有方法仍依赖于均匀的暴力搜索(例如固定最佳-N或自我一致性),这成本高、难以归因,并可能导致边际效益递减的过度思考。我们提出了ODAR-专家,一种自适应路由框架,通过原理性的资源分配优化精度-效率权衡。ODAR使用基于 amortized active inference 的难度估计器,动态地在启发式快速代理和 deliberative 慢速代理之间路由查询。我们进一步引入了一种基于自由能原理、风险敏感的融合机制,通过最小化变分自由能目标选择答案,平衡对数似然与epistemic不确定性(变分熵),作为对异质候选物进行随意投票的原理性替代方案。在23个基准测试中的广泛评估显示,结果具有强且一致的提升,包括在MATH上达到98.2%的准确率,在Humanity's Last Exam (HLE)上达到54.8%的准确率,同时在计算匹配设置下改进了计算-准确率前沿。我们还验证了在完全开源栈(Llama 4 + DeepSeek)上的可重复性,其中ODAR在减少计算成本82%的同时超越了同质采样策略。总体而言,我们的结果表明,最优思考扩展需要基于自由能的决策而非简单地增加测试时计算。
英文摘要:
The paradigm of large language model (LLM) reasoning is shifting from parameter scaling to test-time compute scaling, yet many existing approaches still rely on uniform brute-force sampling (for example, fixed best-of-N or self-consistency) that is costly, hard to attribute, and can trigger overthinking with diminishing returns. We propose ODAR-Expert, an adaptive routing framework that optimizes the accuracy-efficiency trade-off via principled resource allocation. ODAR uses a difficulty estimator grounded in amortized active inference to dynamically route queries between a heuristic Fast Agent and a deliberative Slow Agent. We further introduce a free-energy-principled, risk-sensitive fusion mechanism that selects answers by minimizing a variational free energy objective, balancing log-likelihood with epistemic uncertainty (varentropy) as a principled alternative to ad hoc voting over heterogeneous candidates. Extensive evaluation across 23 benchmarks shows strong and consistent gains, including 98.2% accuracy on MATH and 54.8% on Humanity's Last Exam (HLE), while improving the compute-accuracy frontier under compute-matched settings. We also validate reproducibility on a fully open-source stack (Llama 4 + DeepSeek), where ODAR surpasses homogeneous sampling strategies while reducing computational costs by 82%. Overall, our results suggest that thinking-optimal scaling requires adaptive resource allocation with free-energy-based decision-making rather than simply increasing test-time compute.
Beyond Classification: Dynamic Adapter Routing for Continual Multimodal Retrieval
NASK National Research Institute(NASK国家研究院);IDEAS Research Institute(IDEAS研究所);Warsaw University of Technology(华沙技术大学);Universitat Autonoma de Barcelona(巴塞罗那自治大学)
Beyond Prompt or Skill? Attribution-Guided Optimization of Modular LLM Programs
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen)(深圳未来智能网络研究院(深圳FNii));Sun Yat-sen University(中山大学)
LEC: Linear Expectation Constraints for Selection-Conditioned Risk Control in Selective Prediction and Routing Systems
University of Electronic Science and Technology of China(电子科技大学);Munich Center for Machine Learning(慕尼黑机器学习中心);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Shandong University(山东大学);Tongji University(同济大学);City University of Hong Kong(香港城市大学)
Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校电子与计算机工程系);AI Innovation Institute, Stony Brook University(石溪大学人工智能创新研究所)
Breaking the MoE LLM Trilemma: Dynamic Expert Clustering with Structured Compression
Guangzhou University(广州大学);Institue of Automation Chinese Academy of Sciences(中国科学院自动化研究所);ByteDance Inc.(字节跳动公司);University of Science and Technology Beijing(北京科技大学)
LoopTrap: Termination Poisoning Attacks on LLM Agents
The State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);Zhejiang University(浙江大学);School of Cyber Science and Engineering(网络安全与工程学院);Southeast University(东南大学)
Department of Statistics and Data Science(统计与数据科学系);Department of Statistics(统计系);Data Science, Southern University of Science(数据科学,南方科技大学);College of Computing(计算学院);College of Computing and Data Science(计算与数据科学学院);School of Statistics(统计学系);School of Statistics and Data Science(统计与数据科学系);School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen(人工智能学院,香港中文大学(深圳));Shenzhen Loop Area Institute(深圳河套学院)