基于能量信息扩散模型的无配对真实感图像去雨
Unpaired Photo-realistic Image Deraining with Energy-informed Diffusion Model
AI总结：
针对无配对图像去雨中残留退化和色彩失真问题，提出能量信息扩散模型UPID-EDM，利用CLIP先验和双一致能量函数指导反向采样，有效去除雨纹并保留图像内容，在监督与无参考指标上超越现有方法。
摘要：
现有的无配对图像去雨方法在准确捕捉雨天与晴天域之间的区分特征时面临挑战，导致重建图像中出现残留退化和色彩失真。为此，我们提出了一种用于无配对真实感图像去雨的能量信息扩散模型（UPID-EDM）。首先，我们深入研究了对比语言-图像预训练模型（CLIP）中蕴含的复杂视觉-语言先验，并证明CLIP先验有助于区分雨天和晴天图像。此外，我们引入了一种双一致能量函数（DEF），在消除与雨相关特征的同时保留与雨无关的特征。该能量函数通过非对应的雨天和晴天图像进行训练。另外，我们采用与雨相关性丢弃能量函数（RDEF）和与雨无关性保留能量函数（RPEF）来指导预训练扩散模型的反向采样过程，在有效去除雨纹的同时保留图像内容。大量实验表明，我们的能量信息模型在监督和无参考指标上均超越了现有的无配对学习方法。
英文摘要：
Existing unpaired image deraining approaches face challenges in accurately capture the distinguishing characteristics between the rainy and clean domains, resulting in residual degradation and color distortion within the reconstructed images. To this end, we propose an energy-informed diffusion model for unpaired photo-realistic image deraining (UPID-EDM). Initially, we delve into the intricate visual-language priors embedded within the contrastive language-image pre-training model (CLIP), and demonstrate that the CLIP priors aid in the discrimination of rainy and clean images. Furthermore, we introduce a dual-consistent energy function (DEF) that retains the rain-irrelevant characteristics while eliminating the rain-relevant features. This energy function is trained by the non-corresponding rainy and clean images. In addition, we employ the rain-relevance discarding energy function (RDEF) and the rain-irrelevance preserving energy function (RPEF) to direct the reverse sampling procedure of a pre-trained diffusion model, effectively removing the rain streaks while preserving the image contents. Extensive experiments demonstrate that our energy-informed model surpasses the existing unpaired learning approaches in terms of both supervised and no-reference metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
