部署在野外的神经网络可能被要求对从与训练数据不同的分布中提取的输入进行预测。大量的工作已经证明，对于神经网络高度自信但错误的输入，很容易找到或合成。生成模型被广泛认为对这种错误的置信度具有鲁棒性，因为对输入特征的密度进行建模可以用于检测新的、分布外的输入。在本文中，我们挑战这个假设。我们发现，基于流的模型、VAE和PixelCNN学习的密度无法区分狗、卡车和马等常见物体的图像（即CIFAR-10）和门牌号的图像（即SVHN），当模型在前者上训练时，后者的可能性更高。此外，我们在配对几个流行的图像数据集时发现了这种现象的证据：FashionMNIST vs MNIST，CelebA vs SVHN，ImageNet vs CIFAR-10 / CIFAR-100 / SVHN。为了研究这种奇怪的行为，我们特别关注基于流的生成模型，因为它们是通过精确的边际似然来训练和评估的。我们发现这种行为持续存在，即使当我们限制流量为常数体积的转换。这些转换承认一些理论分析，我们表明，可能性的差异可以解释的数据和模型曲率的位置和方差。我们的研究结果警告不要使用深度生成模型的密度估计来识别类似于训练分布的输入，直到更好地理解它们对分布外输入的行为。
英文摘要：
A neural network deployed in the wild may be asked to make predictions for inputs that were drawn from a different distribution than that of the training data. A plethora of work has demonstrated that it is easy to find or synthesize inputs for which a neural network is highly confident yet wrong. Generative models are widely viewed to be robust to such mistaken confidence as modeling the density of the input features can be used to detect novel, out-of-distribution inputs. In this paper we challenge this assumption. We find that the density learned by flow-based models, VAEs, and PixelCNNs cannot distinguish images of common objects such as dogs, trucks, and horses (i.e. CIFAR-10) from those of house numbers (i.e. SVHN), assigning a higher likelihood to the latter when the model is trained on the former. Moreover, we find evidence of this phenomenon when pairing several popular image data sets: FashionMNIST vs MNIST, CelebA vs SVHN, ImageNet vs CIFAR-10 / CIFAR-100 / SVHN. To investigate this curious behavior, we focus analysis on flow-based generative models in particular since they are trained and evaluated via the exact marginal likelihood. We find such behavior persists even when we restrict the flows to constant-volume transformations. These transformations admit some theoretical analysis, and we show that the difference in likelihoods can be explained by the location and variances of the data and the model curvature. Our results caution against using the density estimates from deep generative models to identify inputs similar to the training distribution until their behavior for out-of-distribution inputs is better understood.
Probabilistic Circuits That Know What They Don't Know
TU Darmstadt（达姆施塔特工业大学）；Hessian Center for AI (hessian.AI)（黑森人工智能中心）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Centre for Cognitive Science TU Darmstadt（达姆施塔特工业大学认知科学中心）
