arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.11062cs.LGstat.ML
收藏

贝叶斯神经网络中量化对模型不确定性的影响

On the Effects of Quantisation on Model Uncertainty in Bayesian Neural Networks

University College London(伦敦大学学院);Arm(Arm公司)

Martin Ferianc, Partha Maji, Matthew Mattina, Miguel Rodrigues

摘要:

贝叶斯神经网络(BNN)在许多需要伴随不确定性估计的决策领域取得了重大进展。能够在决策时量化不确定性对于理解模型何时过度/不足是至关重要的,因此BNN在自动驾驶、医疗保健和机器人等安全关键型应用中引起了人们的兴趣。然而,BNN在工业实践中并没有得到广泛的应用,主要是因为它们增加了内存和计算成本。在这项工作中,我们研究了通过将32位浮点权重和激活压缩到整数对应物来量化BNN,这已经成功地降低了标准逐点神经网络的计算需求。我们研究了三种类型的量化BNN,我们评估他们在广泛的不同设置下,我们的经验表明,一个统一的量化方案适用于BNN不会大幅降低其质量的不确定性估计。

英文摘要:

Bayesian neural networks (BNNs) are making significant progress in many research areas where decision-making needs to be accompanied by uncertainty estimation. Being able to quantify uncertainty while making decisions is essential for understanding when the model is over-/under-confident, and hence BNNs are attracting interest in safety-critical applications, such as autonomous driving, healthcare, and robotics. Nevertheless, BNNs have not been as widely used in industrial practice, mainly because of their increased memory and compute costs. In this work, we investigate quantisation of BNNs by compressing 32-bit floating-point weights and activations to their integer counterparts, that has already been successful in reducing the compute demand in standard pointwise neural networks. We study three types of quantised BNNs, we evaluate them under a wide range of different settings, and we empirically demonstrate that a uniform quantisation scheme applied to BNNs does not substantially decrease their quality of uncertainty estimation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑