通过双重MCMC教学学习基于能量的模型
Learning Energy-based Model via Dual-MCMC Teaching
AI总结：
本文提出一种联合学习框架，通过双重MCMC教学将EBM、生成器和推断模型集成，使生成器同时匹配EBM与经验数据分布，从而为EBM的MCMC采样提供更优初始化，实现有效且高效的EBM学习。
摘要：
本文研究基于能量的模型（EBM）的基础学习问题。学习EBM可以通过最大似然估计（MLE）来实现，这通常涉及马尔可夫链蒙特卡洛（MCMC）采样，例如朗之万动力学。然而，噪声初始化的朗之万动力学在实践中可能具有挑战性且难以混合。这促使人们探索与生成器模型的联合训练，其中生成器模型作为互补模型来绕过MCMC采样。然而，这种方法可能不如MCMC准确，并导致有偏的EBM学习。虽然生成器也可以作为初始化器模型以实现更好的MCMC采样，但其学习可能是有偏的，因为它仅匹配EBM而无法访问经验训练样本。这种有偏的生成器学习可能限制学习EBM的潜力。为了解决这一问题，我们提出了一种联合学习框架，该框架交织了EBM和互补生成器模型的最大似然学习算法。特别是，生成器模型通过MLE学习以同时匹配EBM和经验数据分布，使其成为EBM的MCMC采样中更具信息量的初始化器。利用观测样本学习生成器通常需要对生成器后验进行推断。为了确保准确且高效的推断，我们采用MCMC后验采样，并引入一个互补推断模型来初始化这种潜在MCMC采样。我们表明，通过两次（双重）MCMC教学，三个独立模型可以无缝集成到我们的联合框架中，从而实现有效且高效的EBM学习。
英文摘要：
This paper studies the fundamental learning problem of the energy-based model (EBM). Learning the EBM can be achieved using the maximum likelihood estimation (MLE), which typically involves the Markov Chain Monte Carlo (MCMC) sampling, such as the Langevin dynamics. However, the noise-initialized Langevin dynamics can be challenging in practice and hard to mix. This motivates the exploration of joint training with the generator model where the generator model serves as a complementary model to bypass MCMC sampling. However, such a method can be less accurate than the MCMC and result in biased EBM learning. While the generator can also serve as an initializer model for better MCMC sampling, its learning can be biased since it only matches the EBM and has no access to empirical training examples. Such biased generator learning may limit the potential of learning the EBM. To address this issue, we present a joint learning framework that interweaves the maximum likelihood learning algorithm for both the EBM and the complementary generator model. In particular, the generator model is learned by MLE to match both the EBM and the empirical data distribution, making it a more informative initializer for MCMC sampling of EBM. Learning generator with observed examples typically requires inference of the generator posterior. To ensure accurate and efficient inference, we adopt the MCMC posterior sampling and introduce a complementary inference model to initialize such latent MCMC sampling. We show that three separate models can be seamlessly integrated into our joint framework through two (dual-) MCMC teaching, enabling effective and efficient EBM learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
