通过扩散模型的可处理引导实现图像修复
Image Inpainting via Tractable Steering of Diffusion Models
AI总结：
本研究提出利用概率电路这类可处理概率模型精确计算约束后验，引导扩散模型去噪过程的图像修复框架，在三个数据集上提升修复质量与语义一致性，仅增约10%计算开销，还支持图像区域语义约束。
摘要：
扩散模型是当前生成逼真图像的最先进技术。然而，在图像修复这类有约束的图像生成任务中，控制采样过程仍然具有挑战性，因为对这类约束进行精确条件建模是难以处理的。现有方法采用多种技术来近似约束后验分布，而本文提出利用可处理概率模型（Tractable Probabilistic Models, TPMs）精确且高效计算约束后验的能力，借助该信号引导扩散模型的去噪过程。具体而言，本文采用了一类表达能力强的TPM，名为概率电路（Probabilistic Circuits, PCs）。在已有研究进展的基础上，我们进一步扩展了PCs的规模，使其能够引导扩散模型的图像生成过程。实验结果表明，我们的方法在三个自然图像数据集（即CelebA-HQ、ImageNet和LSUN）上，能够持续提升修复图像的整体质量和语义一致性，而TPM带来的额外计算开销仅约10%。此外，借助图像编码器和解码器，我们的方法可以轻松接受图像特定区域的语义约束，这为更多受控图像生成任务开辟了潜力。除了提出一种新的约束图像生成框架外，本文还强调了更具可处理性模型的优势，并推动了高表达能力TPM的发展。
英文摘要：
Diffusion models are the current state of the art for generating photorealistic images. Controlling the sampling process for constrained image generation tasks such as inpainting, however, remains challenging since exact conditioning on such constraints is intractable. While existing methods use various techniques to approximate the constrained posterior, this paper proposes to exploit the ability of Tractable Probabilistic Models (TPMs) to exactly and efficiently compute the constrained posterior, and to leverage this signal to steer the denoising process of diffusion models. Specifically, this paper adopts a class of expressive TPMs termed Probabilistic Circuits (PCs). Building upon prior advances, we further scale up PCs and make them capable of guiding the image generation process of diffusion models. Empirical results suggest that our approach can consistently improve the overall quality and semantic coherence of inpainted images across three natural image datasets (i.e., CelebA-HQ, ImageNet, and LSUN) with only $\sim\! 10 \%$ additional computational overhead brought by the TPM. Further, with the help of an image encoder and decoder, our method can readily accept semantic constraints on specific regions of the image, which opens up the potential for more controlled image generation tasks. In addition to proposing a new framework for constrained image generation, this paper highlights the benefit of more tractable models and motivates the development of expressive TPMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
