Out-of-distribution (OoD) detection is a natural downstream task for deep generative models, due to their ability to learn the input probability distribution. There are mainly two classes of approaches for OoD detection using deep generative models, viz., based on likelihood measure and the reconstruction loss. However, both approaches are unable to carry out OoD detection effectively, especially when the OoD samples have smaller variance than the training samples. For instance, both flow based and VAE models assign higher likelihood to images from SVHN when trained on CIFAR-10 images. We use a recently proposed generative model known as neural rendering model (NRM) and derive metrics for OoD. We show that NRM unifies both approaches since it provides a likelihood estimate and also carries out reconstruction in each layer of the neural network. Among various measures, we found the joint likelihood of latent variables to be the most effective one for OoD detection. Our results show that when trained on CIFAR-10, lower likelihood (of latent variables) is assigned to SVHN images. Additionally, we show that this metric is consistent across other OoD datasets. To the best of our knowledge, this is the first work to show consistently lower likelihood for OoD data with smaller variance with deep generative models.
DSRGAN: Explicitly Learning Disentangled Representation of Underlying Structure and Rendering for Image Generation without Tuple Supervision
Sun Yat-sen University（中山大学）；NUDT（国防科技大学）
我们专注于明确学习自然图像生成的解纠缠表示，其中底层空间结构和结构上的渲染可以分别独立控制，但不使用元组监督。该设置非常重要，因为元组监督成本很高，有时甚至不可用。然而，该任务是高度不受约束的，因此是不适定的。为了解决这个问题，我们建议引入一个辅助域，它与目标域共享一个共同的底层结构空间，我们做了一个部分共享的潜在空间假设。其关键思想是鼓励部分共享的潜变量来表示两个域中相似的潜在空间结构，而两个域特定的潜变量将被可替换地安排来分别呈现两个域的渲染。这是通过设计两个并行的生成网络与一个共同的渐进渲染架构（PRA），这限制了两个生成网络的行为，以模型共享的底层结构和模型之间的空间依赖关系渲染和底层结构。因此，我们提出DSRGAN（GANs for Disentangling Underlying Structure and Rendering）来实例化我们的方法。我们还提出了一个定量标准（归一化解缠）来量化解缠。与最先进的方法的比较表明，DSRGAN可以显着优于他们在解缠。
Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering
NVIDIA（英伟达）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）；University of Waterloo（滑铁卢大学）；Stanford University（斯坦福大学）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）
DiffRenderGAN: Addressing Training Data Scarcity in Deep Segmentation Networks for Quantitative Nanomaterial Analysis through Differentiable Rendering and Generative Modelling
Department Artificial Intelligence in Biomedical Engineering（人工智能生物医学工程部门）；Friedrich-Alexander-University Erlangen-Nürnberg（埃朗根-纽伦堡弗里德里希-亚历山大大学）；Pattern Recognition Lab（模式识别实验室）；Institute for Nanotechnology and Correlative Microscopy（纳米技术与相关显微镜研究所）；Lucid Concepts AG（Lucid Concepts公司）；Fraunhofer Institute for Ceramic Technologies and Systems（弗劳恩霍夫陶瓷技术与系统研究所）；Freie Universität Berlin（柏林自由大学）；Max Planck Institute for the Science of Light（马克斯·普朗克光科学研究所）；Julius-Maximilians-University Würzburg（乌尔姆-马克斯·普朗克大学）
