arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.07359cs.LG
收藏

基于Dirichlet元模型的事后不确定性学习

Post-hoc Uncertainty Learning using a Dirichlet Meta-Model

Massachusetts Institute of Technology(麻省理工学院);University of Florida(佛罗里达大学);MIT-IBM Watson AI Lab, IBM Research(MIT-IBM Watson AI实验室,IBM研究院)

Maohao Shen, Yuheng Bu, Prasanna Sattigeri, Soumya Ghosh, Subhro Das, Gregory Wornell

摘要:

众所周知,神经网络在直接使用输出标签分布来生成不确定性度量时存在过度自信的问题。现有的方法主要通过重新训练整个模型来解决这个问题,以施加不确定性量化能力,使得学习的模型可以同时在准确性和不确定性预测方面达到期望的性能。然而,从头开始训练模型在计算上是昂贵的,并且在许多情况下可能不可行。在这项工作中,我们考虑了一个更实用的事后不确定性学习设置,其中给出了一个经过良好训练的基础模型,我们专注于训练第二阶段的不确定性量化任务。我们提出了一种新的贝叶斯元模型来增强预训练模型,使其具有更好的不确定性量化能力,这是有效的,计算效率高。我们提出的方法不需要额外的训练数据,并且足够灵活,可以量化不同的不确定性,并轻松适应不同的应用设置,包括域外数据检测,误分类检测和可信的迁移学习。我们证明了我们提出的元模型方法的灵活性和优越的经验性能,这些应用程序在多个代表性的图像分类基准。

英文摘要:

It is known that neural networks have the problem of being over-confident when directly using the output label distribution to generate uncertainty measures. Existing methods mainly resolve this issue by retraining the entire model to impose the uncertainty quantification capability so that the learned model can achieve desired performance in accuracy and uncertainty prediction simultaneously. However, training the model from scratch is computationally expensive and may not be feasible in many situations. In this work, we consider a more practical post-hoc uncertainty learning setting, where a well-trained base model is given, and we focus on the uncertainty quantification task at the second stage of training. We propose a novel Bayesian meta-model to augment pre-trained models with better uncertainty quantification abilities, which is effective and computationally efficient. Our proposed method requires no additional training data and is flexible enough to quantify different uncertainties and easily adapt to different application settings, including out-of-domain data detection, misclassification detection, and trustworthy transfer learning. We demonstrate our proposed meta-model approach's flexibility and superior empirical performance on these applications over multiple representative image classification benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑