Model-free reinforcement learning (RL) is a powerful tool to learn a broad range of robot skills and policies. However, a lack of policy interpretability can inhibit their successful deployment in downstream applications, particularly when differences in environmental conditions may result in unpredictable behaviour or generalisation failures. As a result, there has been a growing emphasis in machine learning around the inclusion of stronger inductive biases in models to improve generalisation. This paper proposes an alternative strategy, inverse value estimation for interpretable policy certificates (IV-Posterior), which seeks to identify the inductive biases or idealised conditions of operation already held by pre-trained policies, and then use this information to guide their deployment. IV-Posterior uses MaskedAutoregressive Flows to fit distributions over the set of conditions or environmental parameters in which a policy is likely to be effective. This distribution can then be used as a policy certificate in downstream applications. We illustrate the use of IV-Posterior across a two environments, and show that substantial performance gains can be obtained when policy selection incorporates knowledge of the inductive biases that these policies hold.
Reliable Error Estimation for PINNs: Lower and Upper A Posteriori Bounds
Physikalisch-Technische Bundesanstalt (PTB)（德国联邦物理技术研究院）；Technical University of Berlin（柏林工业大学）；Weierstrass Institute for Applied Analysis and Stochastics（魏尔斯特拉斯应用分析与随机研究所）；Eastern Mediterranean University（东地中海大学）
Reinforcement learning studies how to balance exploration and exploitation in real-world systems, optimizing interactions with the world while simultaneously learning how the world operates. One general class of algorithms for such learning is the multi-armed bandit setting. Randomized probability matching, based upon the Thompson sampling approach introduced in the 1930s, has recently been shown to perform well and to enjoy provable optimality properties. It permits generative, interpretable modeling in a Bayesian setting, where prior knowledge is incorporated, and the computed posteriors nat…
Uncertainty quantification in model discovery by distilling interpretable material constitutive models from Gaussian process posteriors
Technische Universität Braunschweig, Institute of Applied Mechanics（Braunschweig 技术大学应用力学研究所）；Technische Universität Braunschweig, Institute for Acoustics（Braunschweig 技术大学声学研究所）
ECPO: Evidence-Coupled Policy Optimization for Verifier-Audited Candidate Ranking
Institute of Information Engineering, Chinese Academy of Sciences（信息工程研究所，中国科学院）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Computer simulations have proven a valuable tool for understanding complex phenomena across the sciences. However, the utility of simulators for modelling and forecasting purposes is often restricted by low data quality, as well as practical limits to model fidelity. In order to circumvent these difficulties, we argue that modellers must treat simulators as idealistic representations of the true data generating process, and consequently should thoughtfully consider the risk of model misspecification. In this work we revisit neural posterior estimation (NPE), a class of algorithms that enable b…
Learning Individual Reproductive Behavior from Aggregate Fertility Rates via Neural Posterior Estimation
Max Planck Institute for Demographic Research（马克斯·普朗克人口研究所以）；Statistics Institute, UDELAR（统计研究所，乌拉尔大学）；Center For Demographic Studies, Autonomous University of Barcelona（人口研究所以，巴塞罗那自治大学）；ICREA, Catalan Institution for Research and Advanced Studies（ICREA，加泰罗尼亚研究与高级研究机构）；Machine Learning in Science, University of Tübingen（科学机器学习，图宾根大学）；Tübingen AI Center（图宾根人工智能中心）；Department Empirical Inference, Max Planck Institute for Intelligent Systems（经验推理部门，马克斯·普朗克智能系统研究所）
Application-driven Validation of Posteriors in Inverse Problems
German Cancer Research Center (DKFZ) Heidelberg（德国癌症研究中心（DKFZ）海德堡分部）；German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging（德国癌症研究中心（DKFZ）海德堡分部亥姆霍兹成像分部）；Interdisciplinary Center for Scientific Computing (IWR)（跨学科科学计算中心（IWR））；German Cancer Research Center (DKFZ) Heidelberg, Interactive Machine Learning Group（德国癌症研究中心（DKFZ）海德堡分部交互式机器学习组）；Department of Informatics, Goethe University Frankfurt（法兰克福大学信息学系）；Department of Medicine, Goethe University Frankfurt（法兰克福大学医学系）；German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；Frankfurt Cancer Institute（法兰克福癌症研究所）
