arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.02720cs.LG
收藏

Laplace近似下的可学习不确定性

Learnable Uncertainty under Laplace Approximations

University of Tübingen(图宾根大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所)

Agustinus Kristiadi, Matthias Hein, Philipp Hennig

摘要:

拉普拉斯近似是用于构造贝叶斯神经网络(BNN)的经典的、计算量小的手段。与其他近似BNN一样,人们不一定期望校准诱导的预测不确定性。在这里,我们开发了一种形式主义,明确“训练”的不确定性,以解耦的方式预测本身。为此,我们为拉普拉斯近似网络引入了不确定性单元:与特定权重结构相关的隐藏单元,可以添加到任何预先训练的点估计网络中。由于它们的权重,这些单元是不活动的-它们不影响预测。但它们的存在改变了损失景观的几何形状(特别是Hessian),从而影响了拉普拉斯近似下网络的不确定性估计。我们表明,这样的单位可以通过一个不确定性感知的目标,提高标准拉普拉斯近似在各种不确定性量化任务的性能进行训练。

英文摘要:

Laplace approximations are classic, computationally lightweight means for constructing Bayesian neural networks (BNNs). As in other approximate BNNs, one cannot necessarily expect the induced predictive uncertainty to be calibrated. Here we develop a formalism to explicitly "train" the uncertainty in a decoupled way to the prediction itself. To this end, we introduce uncertainty units for Laplace-approximated networks: Hidden units associated with a particular weight structure that can be added to any pre-trained, point-estimated network. Due to their weights, these units are inactive -- they do not affect the predictions. But their presence changes the geometry (in particular the Hessian) of the loss landscape, thereby affecting the network's uncertainty estimates under a Laplace approximation. We show that such units can be trained via an uncertainty-aware objective, improving standard Laplace approximations' performance in various uncertainty quantification tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑