Behavior Foundation Models (BFMs) are capable of retrieving high-performing policy for any reward function specified directly at test-time, commonly referred to as zero-shot reinforcement learning (RL). While this is a very efficient process in terms of compute, it can be less so in terms of data: as a standard assumption, BFMs require computing rewards over a non-negligible inference dataset, assuming either access to a functional form of rewards, or significant labeling efforts. To alleviate these limitations, we tackle the problem of task inference purely through interaction with the environment at test-time. We propose OpTI-BFM, an optimistic decision criterion that directly models uncertainty over reward functions and guides BFMs in data collection for task inference. Formally, we provide a regret bound for well-trained BFMs through a direct connection to upper-confidence algorithms for linear bandits. Empirically, we evaluate OpTI-BFM on established zero-shot benchmarks, and observe that it enables successor-features-based BFMs to identify and optimize an unseen reward function in a handful of episodes with minimal compute overhead. Code is available at https://github.com/ThomasRupf/opti-bfm.
Optimistic Reinforcement Learning with Quantile Objectives
Grado Department of Industrial & Systems Engineering, Virginia Tech, Blacksburg, VA 24061, USA(弗吉尼亚理工大学工业与系统工程系);Department of Industrial, Manufacturing, and Systems Engineering, University of Texas at Arlington, Arlington, TX 76019, USA(德克萨斯理工大学工业、制造与系统工程系)
Optimism as Risk-Seeking in Multi-Agent Reinforcement Learning
Laboratory for Information & Decision Systems, Massachusetts Institute of Technology(信息与决策系统实验室,麻省理工学院);Harvard University(哈佛大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
An empirical evaluation of active inference in multi-armed bandits
Technische Universität Dresden(德累斯顿工业大学);Centre for Tactile Internet with Human-in-the-Loop (CeTI), Technische Universität Dresden(德累斯顿工业大学触觉互联网与人机闭环研究中心);Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London(伦敦大学学院马克斯·普朗克计算精神病学与衰老研究中心);Secondmind;University College London(伦敦大学学院)
Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models
Machine Learning in Science, University of Tübingen(图宾根大学机器学习与科学系);Tübingen AI Center(图宾根人工智能中心);Department Empirical Inference, Max Planck Institute for Intelligent Systems(智能系统研究所经验推断部门)
CIDER-FM: Foundation Models for Causal Inference from Diverse Experimental Regimes
University of Cambridge(剑桥大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Prior Labs;Gatsby Computational Neuroscience Unit, University College London(伦敦大学学院盖茨比计算神经科学单元);ELLIS Institute(ELLIS研究所);The Alan Turing Institute(艾伦·图灵研究所)
Towards Fast Coarse-graining and Equation Discovery with Foundation Inference Models
Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔人工智能与机器学习研究所);University of Bonn(波恩大学);Fraunhofer IAIS(弗劳恩霍夫人工智能研究所);JetBrains Research(JetBrains研究)
Designing Large Foundation Models for Efficient Training and Inference: A Survey
University of Wisconsin–Madison(威斯康星大学麦迪逊分校);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Columbia University(哥伦比亚大学);University of Illinois at Urbana–Champaign(伊利诺伊大学厄巴纳-香槟分校);The Ohio State University(俄亥俄州立大学);Iran University of Science and Technology(伊朗科技大学)
SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beijing Key Laboratory of Brain-Inspired General Intelligence Large Model(北京脑启发通用智能大模型重点实验室);Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology(脑认知与脑启发智能技术重点实验室);The Hong Kong Polytechnic University(香港理工大学)
Towards Universal Gene Regulatory Network Inference: Unlocking Generalizable Regulatory Knowledge in Single-cell Foundation Models
Computer Network Information Center, Chinese Academy of Sciences, Beijing, China(中国科学院计算机网络信息中心,北京,中国);Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China(中国科学院大学杭州高等研究院,杭州,中国);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学,北京,中国)
Can Tabular Foundation Models Amortize Statistical Inference?
London School of Economics and Political Science(伦敦政治经济学院);University of Science and Technology of China(中国科学技术大学);Shanghai University of Finance and Economics(上海财经大学);University of Milano-Bicocca(米兰比可卡大学)
Neural Architectures for Amortized Bayesian Inference: Statistical Foundations and Empirical Assessments
Department of Mathematics and Statistics, Indian Institute of Technology Kanpur(数学与统计学系,印度理工学院坎浦尔分校);Marshall School of Business, University of Southern California(马歇尔商学院,南加州大学)