arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1902.10189stat.MLcs.LG
收藏

深度神经网络模型不确定性度量的变分推断

Variational Inference to Measure Model Uncertainty in Deep Neural Networks

Alpen-Adria Universität(阿尔卑斯-亚得里亚大学);CTR Carinthian Tech Research AG(克恩滕州技术研究股份公司)

Konstantin Posch, Jan Steinbrener, Jürgen Pilz

摘要:

我们提出了一种以贝叶斯方式训练深度神经网络的新方法。经典的,即非贝叶斯的深度学习有两个主要缺点,这两个缺点都源于网络参数被认为是确定性的。首先,模型的不确定性无法测量,因此限制了深度学习在许多应用领域的使用;其次,深度神经网络的训练通常会受到过拟合的阻碍。所提出的方法使用变分推断来近似一个棘手的后验分布的基础上正常的前。变分密度的设计方式,网络参数的后验不确定性表示每个网络层,并取决于估计的参数期望值。这样,与非贝叶斯网络相比,只需要优化一些额外的参数。我们将这种贝叶斯方法应用于训练并在MNIST数据集上测试LeNet架构。与经典深度学习相比,测试误差减少了15%。此外,训练的模型包含关于每层中的参数不确定性的信息。我们表明,这些信息可用于计算预测的可信区间,并优化给定训练数据集的网络架构。

英文摘要:

We present a novel approach for training deep neural networks in a Bayesian way. Classical, i.e. non-Bayesian, deep learning has two major drawbacks both originating from the fact that network parameters are considered to be deterministic. First, model uncertainty cannot be measured thus limiting the use of deep learning in many fields of application and second, training of deep neural networks is often hampered by overfitting. The proposed approach uses variational inference to approximate the intractable a posteriori distribution on basis of a normal prior. The variational density is designed in such a way that the a posteriori uncertainty of the network parameters is represented per network layer and depending on the estimated parameter expectation values. This way, only a few additional parameters need to be optimized compared to a non-Bayesian network. We apply this Bayesian approach to train and test the LeNet architecture on the MNIST dataset. Compared to classical deep learning, the test error is reduced by 15%. In addition, the trained model contains information about the parameter uncertainty in each layer. We show that this information can be used to calculate credible intervals for the prediction and to optimize the network architecture for a given training data set.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑