arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.08077cs.CV
收藏

基于扩散的图像到图像翻译:通过提示插值进行噪声校正

Diffusion-Based Image-to-Image Translation by Noise Correction via Prompt Interpolation

Seoul National University(首尔大学)

Junsung Lee, Minsoo Kang, Bohyung Han

AI总结:

本研究提出一种免训练的扩散图像翻译方法,通过提示插值构造噪声校正项修正预训练扩散模型的噪声预测,可轻松集成到现有方法中,在低延迟下实现优异性能并提升现有框架效果。

摘要:

我们提出了一种简单而有效的免训练方法,专门针对基于扩散的图像到图像翻译任务。该方法通过引入噪声校正项,对预训练扩散模型的原始噪声预测网络进行修正。我们将噪声校正项定义为两个噪声预测结果的差值:一个由对源提示嵌入和目标提示嵌入进行渐进插值后的去噪网络计算得到,另一个则是使用源提示嵌入得到的噪声预测结果。最终的噪声预测网络由标准去噪项与噪声校正项线性组合而成,其中前者用于重建必须保留的区域,后者则旨在有效编辑与目标提示相关的感兴趣区域。该方法可以轻松集成到现有的基于扩散模型的图像到图像翻译方法中。大量实验验证了所提技术在低延迟下实现了优异性能,并且与现有框架结合时能够持续提升其效果。

英文摘要:

We propose a simple but effective training-free approach tailored to diffusion-based image-to-image translation. Our approach revises the original noise prediction network of a pretrained diffusion model by introducing a noise correction term. We formulate the noise correction term as the difference between two noise predictions; one is computed from the denoising network with a progressive interpolation of the source and target prompt embeddings, while the other is the noise prediction with the source prompt embedding. The final noise prediction network is given by a linear combination of the standard denoising term and the noise correction term, where the former is designed to reconstruct must-be-preserved regions while the latter aims to effectively edit regions of interest relevant to the target prompt. Our approach can be easily incorporated into existing image-to-image translation methods based on diffusion models. Extensive experiments verify that the proposed technique achieves outstanding performance with low latency and consistently improves existing frameworks when combined with them.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑