Assessing Deep Neural Networks as Probability Estimators
University of Nebraska-Lincoln（内布拉斯加大学林肯分校）；Bayesics, LLC（贝西克斯有限责任公司）
Yu Pan, Kwo-Sen Kuo, Michael L. Rilee, Hongfeng Yu
备注：
Y. Pan, K. Kuo, M. Rilee and H. Yu, "Assessing Deep Neural Networks as Probability Estimators," in 2021 IEEE International Conference on Big Data (Big Data), Orlando, FL, USA, 2021 pp. 1083-1091. doi: 10.1109/BigData52589.2021.9671328
Deep Neural Networks (DNNs) have performed admirably in classification tasks. However, the characterization of their classification uncertainties, required for certain applications, has been lacking. In this work, we investigate the issue by assessing DNNs' ability to estimate conditional probabilities and propose a framework for systematic uncertainty characterization. Denoting the input sample as x and the category as y, the classification task of assigning a category y to a given input x can be reduced to the task of estimating the conditional probabilities p(y|x), as approximated by the DNN at its last layer using the softmax function. Since softmax yields a vector whose elements all fall in the interval (0, 1) and sum to 1, it suggests a probabilistic interpretation to the DNN's outcome. Using synthetic and real-world datasets, we look into the impact of various factors, e.g., probability density f(x) and inter-categorical sparsity, on the precision of DNNs' estimations of p(y|x), and find that the likelihood probability density and the inter-categorical sparsity have greater impacts than the prior probability to DNNs' classification uncertainty.
PQMass: Probabilistic Assessment of the Quality of Generative Models using Probability Mass Estimation
Mila – Québec Artificial Intelligence Institute（魁北克人工智能研究所）；Ciela Institute（Ciela研究所）；Université de Montréal（蒙特利尔大学）；SanboxAQ；CCA – Flatiron Institute（CCA-Flatiron研究所）；University of Edinburgh（爱丁堡大学）；Trottier Space Institute（特罗特里尔空间研究所）
