In Bayesian inference, making deductions about a parameter of interest requires one to sample from or compute an integral against a posterior distribution. A popular method to make these computations cheaper in high-dimensional settings is to replace the posterior with its Laplace approximation (LA), a Gaussian distribution. In this work, we derive a leading order decomposition of the LA error, a powerful technique to analyze the accuracy of the approximation more precisely than was possible before. It allows us to derive the first ever skew correction to the LA which provably improves its accuracy by an order of magnitude in the high-dimensional regime. Our approach also enables us to prove both tighter upper bounds on the standard LA and the first ever lower bounds in high dimensions. In particular, we prove that $d^2\ll n$ is in general necessary for accuracy of the LA, where $d$ is dimension and $n$ is sample size. Finally, we apply our theory in two example models: a Dirichlet posterior arising from a multinomial observation, and logistic regression with Gaussian design. In the latter setting, we prove high probability bounds on the accuracy of the LA and skew-corrected LA in powers of $d/\sqrt n$ alone.
Preferential Bayesian optimisation with Skew Gaussian Processes
Trinity College(都柏林圣三一学院);Dalle Molle Institute for Artificial Intelligence Research (IDSIA) - USI/SUPSI(达勒莫尔人工智能研究所(IDSIA)- 瑞士意大利语区应用艺术与科学大学/瑞士南部应用科学与艺术大学)
Addressing the Inconsistency in Bayesian Deep Learning via Generalized Laplace Approximation
School of Engineering Deakin University Melbourne, VIC, 3216, Australia(工程学院 德堪大学 澳大利亚墨尔本 VIC 3216);Faculty of Mathematics(数学与计算机科学学院);Department of Computing Technologies Swinburne University of Technology Hawthorn, VIC, 3122, Australia(计算技术系 斯winburne技术大学 澳大利亚霍斯顿 VIC 3122)
Beyond Laplace: Closed-form wrapped Gaussian posterior approximations on statistical manifolds
University of Helsinki(赫尔辛基大学);Nanyang Technological University(南洋理工大学);Qutwo;DTU Compute, Technical University of Denmark(丹麦技术大学计算学院);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);University of Cambridge(剑桥大学);The Alan Turing Institute(艾伦·图灵研究所)
A fast, flexible simulation framework for Bayesian adaptive designs -- the R package BATSS
MRC Biostatistics Unit, University of Cambridge(剑桥大学医学研究委员会生物统计学单位);School of Public Health and Preventive Medicine, Monash University(莫纳什大学公共卫生与预防医学院);Faculty of Informatics and Data Science, University of Regensburg(雷根斯堡大学信息科学与数据科学学院)