能量校准VAE与测试时免费午餐
Energy-Calibrated VAE with Test Time Free Lunch
AI总结：
提出能量校准VAE（EC-VAE），利用条件能量模型校准生成方向，训练时使用自适应权重，避免测试时MCMC采样，在图像生成和零样本恢复中取得竞争力性能。
摘要：
在本文中，我们提出了一种新颖的生成模型，利用条件能量模型（EBM）来增强变分自编码器（VAE），称为能量校准VAE（EC-VAE）。具体来说，VAE常常因缺乏针对生成方向生成样本的定制训练而产生模糊的生成样本。另一方面，EBM可以生成高质量样本，但需要昂贵的马尔可夫链蒙特卡洛（MCMC）采样。为解决这些问题，我们在训练期间引入条件EBM来校准VAE的生成方向，而无需在测试时进行生成。特别地，我们基于输入数据和校准样本，使用自适应权重训练EC-VAE，以提高效率并避免测试时的MCMC采样。此外，我们将EC-VAE的校准思想扩展到变分学习和归一化流，并通过神经传输先验和范围-零理论将EC-VAE应用于零样本图像恢复的额外应用。我们通过两个应用评估所提出的方法，包括图像生成和零样本图像恢复，实验结果表明，我们的方法在单步非对抗生成中取得了有竞争力的性能。我们的代码可在https://github.com/DJ-LYH/EC-VAE获取。
英文摘要：
In this paper, we propose a novel generative model that utilizes a conditional Energy-Based Model (EBM) for enhancing Variational Autoencoder (VAE), termed Energy-Calibrated VAE (EC-VAE). Specifically, VAEs often suffer from blurry generated samples due to the lack of a tailored training on the samples generated in the generative direction. On the other hand, EBMs can generate high-quality samples but require expensive Markov Chain Monte Carlo (MCMC) sampling. To address these issues, we introduce a conditional EBM for calibrating the generative direction of VAE during training, without requiring it for the generation at test time. In particular, we train EC-VAE upon both the input data and the calibrated samples with adaptive weight to enhance efficacy while avoiding MCMC sampling at test time. Furthermore, we extend the calibration idea of EC-VAE to variational learning and normalizing flows, and apply EC-VAE to an additional application of zero-shot image restoration via neural transport prior and range-null theory. We evaluate the proposed method with two applications, including image generation and zero-shot image restoration, and the experimental results show that our method achieves competitive performance over single-step non-adversarial generation. Our code is available at https://github.com/DJ-LYH/EC-VAE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
