Reduce, Reuse, Recycle: 基于能量的扩散模型与MCMC的组合生成
Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC
AI总结：
针对现有扩散模型组合生成失效的问题，提出基于MCMC的新型采样器及扩散模型的能量参数化方法，在多类组合生成任务中实现性能显著提升。
摘要：
扩散模型自问世以来，已迅速成为众多领域生成式建模的主流方法。它们可被解释为学习随时间变化的对数概率密度函数序列的梯度，这一解释推动了基于分类器的引导和无分类器引导方法的发展，用于对扩散模型进行事后控制。本研究基于扩散模型的得分解释，探索在组合生成与引导任务中对扩散模型进行条件设置、修改和复用的替代方案。我们重点研究了现有技术下某些类型的组合生成失效的原因，并提出了若干解决方案。研究表明，失效的根源在于采样器而非模型本身，为此我们受MCMC（马尔可夫链蒙特卡洛）启发提出了新型采样器，可实现成功的组合生成。此外，我们提出了扩散模型的基于能量的参数化方式，支持使用新的组合算子以及更复杂的Metropolis校正采样器。有趣的是，我们发现这些采样器在分类器引导的ImageNet建模、组合式文本到图像生成等广泛问题中，均带来了组合生成性能的显著提升。
英文摘要：
Since their introduction, diffusion models have quickly become the prevailing approach to generative modeling in many domains. They can be interpreted as learning the gradients of a time-varying sequence of log-probability density functions. This interpretation has motivated classifier-based and classifier-free guidance as methods for post-hoc control of diffusion models. In this work, we build upon these ideas using the score-based interpretation of diffusion models, and explore alternative ways to condition, modify, and reuse diffusion models for tasks involving compositional generation and guidance. In particular, we investigate why certain types of composition fail using current techniques and present a number of solutions. We conclude that the sampler (not the model) is responsible for this failure and propose new samplers, inspired by MCMC, which enable successful compositional generation. Further, we propose an energy-based parameterization of diffusion models which enables the use of new compositional operators and more sophisticated, Metropolis-corrected samplers. Intriguingly we find these samplers lead to notable improvements in compositional generation across a wide set of problems such as classifier-guided ImageNet modeling and compositional text-to-image generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
