基于潜在二元变量与归一化流的贝叶斯神经网络稀疏化方法
Sparsifying Bayesian neural networks with latent binary variables and normalizing flows
AI总结：
针对贝叶斯神经网络过拟合与不确定性估计需求，对潜在二元贝叶斯神经网络提出局部重参数化技巧加速和归一化流变分后验两项扩展，提升预测性能与网络稀疏性，得到更真实的不确定性估计。
摘要：
人工神经网络（ANNs）是一类强大的机器学习方法，被应用于面部识别、机器翻译、癌症诊断等诸多现代场景。ANNs的一个普遍问题是其通常包含数百万乃至数十亿可训练参数，因此容易对训练数据产生过拟合，这在需要可靠不确定性估计的应用中问题尤为突出。贝叶斯神经网络（BNN）通过纳入参数不确定性，能够改善这一问题。此外，潜在二元贝叶斯神经网络（LBBNN，latent binary Bayesian neural networks）还通过允许权重被启用或关闭来考虑结构不确定性，实现了权重与结构联合空间中的推断。本文对LBBNN方法提出两项扩展：其一，利用局部重参数化技巧（LRT，local reparametrization trick）直接对隐藏单元采样，得到了计算效率更高的算法；更关键的是，通过对LBBNN参数的变分后验分布施加归一化流，网络能够学习到比平均场高斯分布更灵活的变分后验分布。实验结果表明，该方法相比LBBNN方法提升了预测性能，同时得到了更稀疏的网络。本文开展了两项模拟研究：第一项研究考虑逻辑回归场景下的变量选择问题，更灵活的变分分布带来了性能提升；第二项研究基于二维高斯分布生成的数据，对预测不确定性进行比较，结果表明本文提出的贝叶斯方法能够给出更符合实际的预测不确定性估计。
英文摘要：
Artificial neural networks (ANNs) are powerful machine learning methods used in many modern applications such as facial recognition, machine translation, and cancer diagnostics. A common issue with ANNs is that they usually have millions or billions of trainable parameters, and therefore tend to overfit to the training data. This is especially problematic in applications where it is important to have reliable uncertainty estimates. Bayesian neural networks (BNN) can improve on this, since they incorporate parameter uncertainty. In addition, latent binary Bayesian neural networks (LBBNN) also take into account structural uncertainty by allowing the weights to be turned on or off, enabling inference in the joint space of weights and structures. In this paper, we will consider two extensions to the LBBNN method: Firstly, by using the local reparametrization trick (LRT) to sample the hidden units directly, we get a more computationally efficient algorithm. More importantly, by using normalizing flows on the variational posterior distribution of the LBBNN parameters, the network learns a more flexible variational posterior distribution than the mean field Gaussian. Experimental results show that this improves predictive power compared to the LBBNN method, while also obtaining more sparse networks. We perform two simulation studies. In the first study, we consider variable selection in a logistic regression setting, where the more flexible variational distribution leads to improved results. In the second study, we compare predictive uncertainty based on data generated from two-dimensional Gaussian distributions. Here, we argue that our Bayesian methods lead to more realistic estimates of predictive uncertainty.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
