arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.04392cs.LGcs.AI
收藏

贝叶斯神经网络的模型结构自适应

Model Architecture Adaption for Bayesian Neural Networks

University of Cambridge(剑桥大学)

Duo Wang, Yiren Zhao, Ilia Shumailov, Robert Mullins

摘要:

贝叶斯神经网络(BNN)提供了一个数学基础的框架来量化模型预测的不确定性,但训练和推理的计算成本过高。在这项工作中,我们展示了一种新的网络架构搜索(NAS),它优化了BNN的准确性和不确定性,同时减少了推理延迟。不同于典型的NAS,仅优化分布的可能性,所提出的计划搜索的不确定性性能使用的分布和分布外的数据。我们的方法能够搜索网络中贝叶斯层的正确位置。在我们的实验中,搜索的模型显示出与最先进的(深度集成)相比相当的不确定性量化能力和准确性。此外,与许多流行的BNN基线相比,搜索模型仅使用一小部分运行时,与MCDropout和深度集成相比,CIFAR 10数据集上的推理运行时成本分别降低了$2.98 \times$和$2.92 \times$。

英文摘要:

Bayesian Neural Networks (BNNs) offer a mathematically grounded framework to quantify the uncertainty of model predictions but come with a prohibitive computation cost for both training and inference. In this work, we show a novel network architecture search (NAS) that optimizes BNNs for both accuracy and uncertainty while having a reduced inference latency. Different from canonical NAS that optimizes solely for in-distribution likelihood, the proposed scheme searches for the uncertainty performance using both in- and out-of-distribution data. Our method is able to search for the correct placement of Bayesian layer(s) in a network. In our experiments, the searched models show comparable uncertainty quantification ability and accuracy compared to the state-of-the-art (deep ensemble). In addition, the searched models use only a fraction of the runtime compared to many popular BNN baselines, reducing the inference runtime cost by $2.98 \times$ and $2.92 \times$ respectively on the CIFAR10 dataset when compared to MCDropout and deep ensemble.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑