发表机构
Scientific Computing Center, Karlsruhe Institute of Technology; Helmholtz AI（科学计算中心，卡尔斯鲁厄理工学院; 亥姆霍兹人工智能中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究贝叶斯神经网络中似然分布假设，通过比较如偏态或重尾等多种假设，发现学生t分布在回归任务中比高斯似然分布预测性能更好，与数据及MLP架构无关，还可能缩短训练时间且易实现。
AI 中文摘要
在贝叶斯神经网络（BNN）中，变分推理是一种广泛采用的以分布方式对不确定性建模的框架，证据下界（ELBO）作为标准目标函数。几个分布对ELBO损失有贡献，如先验、近似后验和似然分布。通常这些分布都用高斯分布近似，因其易于计算、允许重参数化梯度并提供封闭形式的训练损失。但有研究指出该假设可能不成立，存在模型错误指定风险。此前针对先验提出了替代分布，而似然分布选择的影响未被探索。本文旨在通过研究似然分布的替代假设是否能优于常用高斯分布来填补这一空白。我们使用标准多层感知器（MLP）在人工和真实世界数据集的回归任务中比较了几种似然分布假设，如偏态或重尾分布。结果表明，学生t分布比高斯似然分布具有更好的预测性能，与数据分布和MLP架构（深度和宽度）无关。在某些情况下，学生t分布还能缩短训练时间且易于实现。
英文摘要
In Bayesian neural networks (BNNs), variational inference is a widely adopted framework for modeling uncertainty in a distributional way, with the evidence lower bound (ELBO) serving as the standard objective function. Several distributions contribute to the ELBO loss, such as the prior, approximated posterior, and likelihood distribution. Typically, these distributions are all approximated by a Gaussian distribution, since it is easy to compute, allows for reparameterized gradients, and provides a closed-form loss for training. However, several works have highlighted that this assumption may not generally hold, posing the risk of model misspecification. Alternative distributions have been proposed for the prior specifically, while the effect of distribution choice on the likelihood distribution remains unexplored. In this work, our aim is to close this gap by investigating whether alternative assumptions for the likelihood distribution can outperform the commonly used Gaussian. We compare several likelihood distribution assumptions, such as skewed or heavy-tailed, across regression tasks on both artificial and real-world datasets using standard multilayer perceptrons (MLPs). Our findings demonstrate that Student's t yields better predictive performance than a Gaussian likelihood distribution, independent of the data distribution and MLP architecture (depth and width). In some cases, Student's t can also lead to shorter training times, while still being easy to implement.
