Large-scale linear models are ubiquitous throughout machine learning, with contemporary application as surrogate models for neural network uncertainty quantification; that is, the linearised Laplace method. Alas, the computational cost associated with Bayesian linear models constrains this method's application to small networks, small output spaces and small datasets. We address this limitation by introducing a scalable sample-based Bayesian inference method for conjugate Gaussian multi-output linear models, together with a matching method for hyperparameter (regularisation) selection. Furthermore, we use a classic feature normalisation method (the g-prior) to resolve a previously highlighted pathology of the linearised Laplace method. Together, these contributions allow us to perform linearised neural network inference with ResNet-18 on CIFAR100 (11M parameters, 100 outputs x 50k datapoints), with ResNet-50 on Imagenet (50M parameters, 1000 outputs x 1.2M datapoints) and with a U-Net on a high-resolution tomographic reconstruction task (2M parameters, 251k output~dimensions).
Adapting the Linearised Laplace Model Evidence for Modern Deep Learning
University of Cambridge(剑桥大学);University of Alberta(阿尔伯塔大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);University College London(伦敦大学学院);University of Amsterdam(阿姆斯特丹大学)
Corrected Integrated Laplace Approximation for Bayesian Inference in Latent Gaussian Models
Manning College of Information and Computer Sciences University of Massachusetts Amherst(信息与计算机科学学院 马萨诸塞大学阿姆赫斯特分校);Department of Statistics University of British Columbia(统计学系 不列颠哥伦比亚大学)
Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantification
Rational Intelligence Lab, CISPA Helmholtz Center for Information Security Department of Statistics, LMU Munich(理性智能实验室,CISPA海德堡信息安全中心统计学系,慕尼黑大学);Department of Statistics, LMU Munich(统计学系,慕尼黑大学);Department of Computer Science, The University of Manchester(计算机科学系,曼彻斯特大学)
Reparameterization invariance in approximate Bayesian inference
Technical University of Denmark(丹麦技术大学);University of Cambridge(剑桥大学);Karolinska Institutet(卡罗林斯卡学院);University of Tübingen(蒂宾根大学);Tübingen AI Center(蒂宾根人工智能中心)
Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Sampling
Nanyang Technological University(南洋理工大学);Eötvös Loránd University (ELTE)(厄特沃什·罗兰大学(ELTE));HUN-REN Institute for Computer Science and Control (SZTAKI)(匈牙利国家研究网络计算机科学与控制研究所(SZTAKI))
Laplace-Fisher Gate Identities for Optimal Matrix-Gated Blended Score Estimation
The Oden Institute for Computational Engineering and Sciences(奥登计算工程与科学研究院);The University of Texas at Austin(德克萨斯大学奥斯汀分校);The Department of Aerospace Engineering & Engineering Mechanics(航空航天工程与机械系)
Beyond Laplace: Closed-form wrapped Gaussian posterior approximations on statistical manifolds
University of Helsinki(赫尔辛基大学);Nanyang Technological University(南洋理工大学);Qutwo;DTU Compute, Technical University of Denmark(丹麦技术大学计算学院);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);University of Cambridge(剑桥大学);The Alan Turing Institute(艾伦·图灵研究所)
Bayesian Reasoning for Physics Informed Neural Networks
Institute for Theoretical Physics, University of Wroc aw(沃拉夫大学理论物理研究所);Institute of Low Temperature and Structure Research(低温与结构研究所);Polish Academy of Sciences(波兰科学院)
Leveraging Sparsity to Improve No-U-Turn Sampling Efficiency for Hierarchical Bayesian Models
Alaska Fisheries Science Center(阿拉斯加渔业科学中心);NOAA Fisheries(美国国家海洋和大气管理局渔业局);DTU Aqua(丹麦技术大学水科学研究所);Technical University of Denmark(丹麦技术大学);Center for Computational Mathematics(计算数学中心);Flatiron Institute(平顿研究所)
VITS : Variational Inference Thompson Sampling for contextual bandits
CMAP, CNRS, Ecole Polytechnique, Institut Polytechnique de Paris(法国综合理工学院CMAP、法国国家科学研究中心、巴黎综合理工学院、巴黎理工学院);Inria Paris(法国国家信息与自动化研究所巴黎分部);Centre de Recherche des Cordeliers, INSERM, Universite de Paris, Sorbonne Universite(科尔德里耶研究中心、法国国家健康与医学研究院、巴黎大学、索邦大学)