通过能量精炼与流形投影学习：一种协同EBM-AE框架

中文摘要 AI 辅助

能量基模型（EBMs）通过学习一个能量景观来提供生成建模的灵活框架，该能量景观为真实样本分配低能量值，而为不太可能的观测分配较高能量。尽管具有理论吸引力，但由于Langevin采样的计算成本以及高效探索所学数据流形的困难，训练EBMs仍然具有挑战性。在这项工作中，我们提出了一种协同的能量基模型与自编码器（EBM-AE）框架，该框架将能量精炼与流形投影相结合。所提出的方法联合训练一个EBM与一个去噪自编码器，并引入一种迭代的EBM→AE→EBM采样过程，其中Langevin动力学和自编码器投影交替精炼生成的样本。在该框架内，自编码器充当流形投影算子，用于正则化采样轨迹，而EBM则执行基于能量的精炼，朝向所学分布的低能量区域。在MNIST数据集上进行的大量实验表明，与传统的自编码器相比，联合EBM-AE训练显著提高了生成质量。除了无条件生成之外，我们还评估了所提出框架在涉及结构化掩码和随机掩码的图像修复任务上的表现。结果表明，流形投影提供了大部分的重建能力，而随着重建问题变得更加具有挑战性，最终的基于能量的精炼变得越来越有益。综合来看，这些结果表明，将流形投影与能量最小化相结合，为生成、重建和分布外检测提供了一个有效且可解释的框架，同时为能量基建模和表示学习的互补作用提供了新的见解。

英文摘要

Energy-Based Models (EBMs) provide a flexible framework for generative modeling by learning an energy landscape that assigns low energy values to realistic samples and higher energies to unlikely observations. Despite their theoretical appeal, training EBMs remains challenging due to the computational cost of Langevin sampling and the difficulty of efficiently exploring the learned data manifold. In this work, we propose a cooperative Energy-Based Model and Autoencoder (EBM-AE) framework that combines energy-based refinement with manifold projection. The proposed approach jointly trains an EBM with a denoising autoencoder and introduces an iterative EBM$\rightarrow$AE$\rightarrow$EBM sampling procedure in which Langevin dynamics and autoencoder projection alternately refine generated samples. Within this framework, the autoencoder acts as a manifold projection operator that regularizes sampling trajectories, while the EBM performs energy-based refinement toward low-energy regions of the learned distribution. Extensive experiments conducted on the MNIST dataset demonstrate that joint EBM-AE training substantially improves generation quality compared with a conventional autoencoder. Beyond unconditional generation, we evaluate the proposed framework on image inpainting tasks involving structured and random masks. The results show that manifold projection provides the majority of the reconstruction capability, whereas the final energy-based refinement becomes increasingly beneficial as the reconstruction problem becomes more challenging. Taken together, the results indicate that combining manifold projection and energy minimization provides an effective and interpretable framework for generation, reconstruction, and out-of-distribution detection, while offering new insights into the complementary roles of energy-based modeling and representation learning.

发表机构

• Hydro-Québec（魁北克水电公司）

机构由 AI 辅助整理，请以论文原文为准。