arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.03288cs.LGstat.ML
收藏

如何训练基于能量的模型

How to Train Your Energy-Based Models

Stanford University(斯坦福大学);Google Research(谷歌研究院)

Yang Song, Diederik P. Kingma

摘要:

基于能量的模型(EBM),也称为非归一化概率模型,指定概率密度或质量函数,直到未知的归一化常数。与大多数其他概率模型不同,EBM不限制归一化常数的可处理性,因此对参数化更灵活,并且可以模拟更具表现力的概率分布族。然而,EBM的未知归一化常数使得训练特别困难。我们的目标是提供一个友好的介绍现代循证医学培训的方法。我们首先解释使用马尔可夫链蒙特卡罗(MCMC)的最大似然训练,然后详细说明无MCMC的方法,包括分数匹配(SM)和噪声压缩估计(NCE)。我们强调这三种方法之间的理论联系,并结束了一个简短的调查替代培训方法,这仍然是积极的研究。我们的教程针对的是对生成模型有基本了解的观众,他们希望应用EBM或在这个方向上开始研究项目。

英文摘要:

Energy-Based Models (EBMs), also known as non-normalized probabilistic models, specify probability density or mass functions up to an unknown normalizing constant. Unlike most other probabilistic models, EBMs do not place a restriction on the tractability of the normalizing constant, thus are more flexible to parameterize and can model a more expressive family of probability distributions. However, the unknown normalizing constant of EBMs makes training particularly difficult. Our goal is to provide a friendly introduction to modern approaches for EBM training. We start by explaining maximum likelihood training with Markov chain Monte Carlo (MCMC), and proceed to elaborate on MCMC-free approaches, including Score Matching (SM) and Noise Constrastive Estimation (NCE). We highlight theoretical connections among these three approaches, and end with a brief survey on alternative training methods, which are still under active research. Our tutorial is targeted at an audience with basic understanding of generative models who want to apply EBMs or start a research project in this direction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑