It is known that neural networks have the problem of being over-confident when directly using the output label distribution to generate uncertainty measures. Existing methods mainly resolve this issue by retraining the entire model to impose the uncertainty quantification capability so that the learned model can achieve desired performance in accuracy and uncertainty prediction simultaneously. However, training the model from scratch is computationally expensive and may not be feasible in many situations. In this work, we consider a more practical post-hoc uncertainty learning setting, where a well-trained base model is given, and we focus on the uncertainty quantification task at the second stage of training. We propose a novel Bayesian meta-model to augment pre-trained models with better uncertainty quantification abilities, which is effective and computationally efficient. Our proposed method requires no additional training data and is flexible enough to quantify different uncertainties and easily adapt to different application settings, including out-of-domain data detection, misclassification detection, and trustworthy transfer learning. We demonstrate our proposed meta-model approach's flexibility and superior empirical performance on these applications over multiple representative image classification benchmarks.
FHGE: A Fast Heterogeneous Graph Embedding with Ad-hoc Meta-paths
School of Computer Science, Fudan University(复旦大学计算机科学学院);School of Software, Fudan University(复旦大学软件学院);Shanghai Key Laboratory of Data Science(上海数据科学重点实验室)
Australian Institute for Machine Learning(澳大利亚机器学习研究所);University of Adelaide(阿德莱德大学);University of Liverpool(利物浦大学);Faculty of Information Technology(信息技术学院);Monash University(莫纳什大学)