arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.04994stat.MLcs.AIcs.LG
收藏

大型线性模型的基于采样的推断及其在线性化拉普拉斯中的应用

Sampling-based inference for large linear models, with application to linearised Laplace

University of Cambridge(剑桥大学);University of Amsterdam(阿姆斯特丹大学);University College London(伦敦大学学院);University of Alberta(阿尔伯塔大学)

Javier Antorán, Shreyas Padhy, Riccardo Barbano, Eric Nalisnick, David Janz, José Miguel Hernández-Lobato

摘要:

大规模线性模型在机器学习中无处不在,当代的应用是作为神经网络不确定性量化的替代模型;即线性化拉普拉斯方法。唉,与贝叶斯线性模型相关的计算成本限制了这种方法在小网络、小输出空间和小数据集上的应用。我们通过引入共轭高斯多输出线性模型的可扩展的基于样本的贝叶斯推断方法,以及超参数(正则化)选择的匹配方法来解决这个限制。此外,我们使用经典的特征归一化方法(g先验)来解决线性拉普拉斯方法的先前突出显示的病理。总之,这些贡献使我们能够在CIFAR 100上使用ResNet-18(11 M参数,100个输出x 50 k数据点),在Imagenet上使用ResNet-50(50 M参数,1000个输出x 1.2M数据点)以及在高分辨率断层重建任务上使用U-Net(2 M参数,251 k输出尺寸)执行线性神经网络推理。

英文摘要:

Large-scale linear models are ubiquitous throughout machine learning, with contemporary application as surrogate models for neural network uncertainty quantification; that is, the linearised Laplace method. Alas, the computational cost associated with Bayesian linear models constrains this method's application to small networks, small output spaces and small datasets. We address this limitation by introducing a scalable sample-based Bayesian inference method for conjugate Gaussian multi-output linear models, together with a matching method for hyperparameter (regularisation) selection. Furthermore, we use a classic feature normalisation method (the g-prior) to resolve a previously highlighted pathology of the linearised Laplace method. Together, these contributions allow us to perform linearised neural network inference with ResNet-18 on CIFAR100 (11M parameters, 100 outputs x 50k datapoints), with ResNet-50 on Imagenet (50M parameters, 1000 outputs x 1.2M datapoints) and with a U-Net on a high-resolution tomographic reconstruction task (2M parameters, 251k output~dimensions).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑