变分自编码器全局最优解的表征与避免
Characterizing and Avoiding Problematic Global Optima of Variational Autoencoders
摘要：
变分自动编码器（VAE）是由两个组件组成的深度生成潜在变量模型：生成模型，通过在潜在空间上转换分布p（z）来捕获数据分布p（x），以及推断每个数据点可能的潜在代码的推断模型（Kingma和Welling，2013）。最近的研究表明，传统的训练方法往往会产生违反建模要求的解决方案：（1）学习的生成模型捕获了观察到的数据分布，但忽略了潜在代码，导致代码不能代表数据（例如van den Oord et al.（2017）; Kim et al.（2018））;（2）学习的潜在代码的聚合与先验p（z）不匹配。这种不匹配意味着学习的生成模型将无法使用p（z）的样本生成真实的数据（例如Makhzani et al.（2015）; Tomczak and Welling（2017））。在本文中，我们证明了这两个问题源于这样一个事实，即VAE训练目标的全局最优值往往对应于不期望的解决方案。我们的分析建立在两个观察基础上：（1）生成模型是不可识别的-存在许多生成模型可以同样很好地解释数据，每个模型都具有不同的（并且可能不需要的）属性;（2）VAE目标中的偏见-VAE目标可能更喜欢解释数据不佳但具有易于近似的后验的生成模型。我们提出了一种新的推理方法，LiBI，减轻在我们的分析中发现的问题。在合成数据集上，我们证明了LiBI可以学习生成模型，这些模型可以捕获数据分布和推理模型，从而在传统方法难以做到这一点时更好地满足建模假设。
英文摘要：
Variational Auto-encoders (VAEs) are deep generative latent variable models consisting of two components: a generative model that captures a data distribution p(x) by transforming a distribution p(z) over latent space, and an inference model that infers likely latent codes for each data point (Kingma and Welling, 2013). Recent work shows that traditional training methods tend to yield solutions that violate modeling desiderata: (1) the learned generative model captures the observed data distribution but does so while ignoring the latent codes, resulting in codes that do not represent the data (e.g. van den Oord et al. (2017); Kim et al. (2018)); (2) the aggregate of the learned latent codes does not match the prior p(z). This mismatch means that the learned generative model will be unable to generate realistic data with samples from p(z)(e.g. Makhzani et al. (2015); Tomczak and Welling (2017)). In this paper, we demonstrate that both issues stem from the fact that the global optima of the VAE training objective often correspond to undesirable solutions. Our analysis builds on two observations: (1) the generative model is unidentifiable - there exist many generative models that explain the data equally well, each with different (and potentially unwanted) properties and (2) bias in the VAE objective - the VAE objective may prefer generative models that explain the data poorly but have posteriors that are easy to approximate. We present a novel inference method, LiBI, mitigating the problems identified in our analysis. On synthetic datasets, we show that LiBI can learn generative models that capture the data distribution and inference models that better satisfy modeling assumptions when traditional methods struggle to do so.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
