arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.05033stat.MLcs.LG
收藏

广义能量模型

Generalized Energy Based Models

Gatsby Computational Neuroscience Unit, University College London(伦敦大学学院盖茨比计算神经科学单元)

Michael Arbel, Liang Zhou, Arthur Gretton

摘要:

我们介绍了广义能量为基础的模型(GEBM)的生成建模。这些模型结合了两个经过训练的组件:一个基本分布(通常是隐式模型),它可以学习高维空间中具有低内在维度的数据的支持;以及一个能量函数,用于细化学习支持的概率质量。能量函数和基共同构成最终模型,这与GANs不同,GANs仅保留基分布(“生成器”)。GEBM通过在学习能量和基础之间交替进行训练。我们证明了这两个训练阶段都是定义明确的:通过最大化广义似然来学习能量,并且由此产生的基于能量的损失为学习基础提供了信息梯度。来自训练模型的潜在空间上的后验样本可以通过MCMC获得,从而在该空间中找到产生更好质量样本的区域。从经验上讲,图像生成任务上的GEBM样本的质量要比单独学习生成器的质量好得多,这表明在所有其他条件相同的情况下,GEBM将优于相同复杂度的GAN。当使用标准化流作为基本度量时,GEBM成功地完成了密度建模任务,返回了与相同网络的直接极大似然相当的性能。

英文摘要:

We introduce the Generalized Energy Based Model (GEBM) for generative modelling. These models combine two trained components: a base distribution (generally an implicit model), which can learn the support of data with low intrinsic dimension in a high dimensional space; and an energy function, to refine the probability mass on the learned support. Both the energy function and base jointly constitute the final model, unlike GANs, which retain only the base distribution (the "generator"). GEBMs are trained by alternating between learning the energy and the base. We show that both training stages are well-defined: the energy is learned by maximising a generalized likelihood, and the resulting energy-based loss provides informative gradients for learning the base. Samples from the posterior on the latent space of the trained model can be obtained via MCMC, thus finding regions in this space that produce better quality samples. Empirically, the GEBM samples on image-generation tasks are of much better quality than those from the learned generator alone, indicating that all else being equal, the GEBM will outperform a GAN of the same complexity. When using normalizing flows as base measures, GEBMs succeed on density modelling tasks, returning comparable performance to direct maximum likelihood of the same networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑