arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.00307cs.CV
收藏

用于高效图像到图像转换的扩散模型翻译器

A Diffusion Model Translator for Efficient Image-to-Image Translation

Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学);Texas A&M University(德克萨斯农工大学)

Mengfei Xia, Yu Zhou, Ran Yi, Yong-Jin Liu, Wenping Wang

AI总结:

提出轻量级扩散模型翻译器(DMT),通过理论证明仅在某中间步骤进行领域分布转换即可完成I2I任务,并设计自动选择时间步策略,在质量和效率上超越现有方法。

摘要:

由于实际应用的需求,将扩散模型应用于图像到图像转换(I2I)近期受到了越来越多的关注。以往的方法将来自源图像的信息注入到每个去噪步骤中进行迭代细化,从而导致了耗时的实现过程。我们提出了一种高效的方法,为扩散模型配备一个轻量级的翻译器,称为 Diffusion Model Translator (DMT),以完成 I2I 任务。具体而言,我们首先提供了理论证明,即在将开创性的 DDPM 工作用于 I2I 任务时,仅在某些中间步骤将分布从一个领域转换到另一个领域既是可行的也是充分的。我们进一步观察到,转换性能高度依赖于为领域转换所选择的时间步,因此提出了一种实用策略,为给定任务自动选择合适的时间步。我们在一系列 I2I 应用上评估了我们的方法,包括图像风格化、图像着色、分割到图像以及草图到图像,以验证其有效性和通用性。比较表明,我们的 DMT 在质量和效率上都超越了现有方法。代码将公开发布。

英文摘要:

Applying diffusion models to image-to-image translation (I2I) has recently received increasing attention due to its practical applications. Previous attempts inject information from the source image into each denoising step for an iterative refinement, thus resulting in a time-consuming implementation. We propose an efficient method that equips a diffusion model with a lightweight translator, dubbed a Diffusion Model Translator (DMT), to accomplish I2I. Specifically, we first offer theoretical justification that in employing the pioneering DDPM work for the I2I task, it is both feasible and sufficient to transfer the distribution from one domain to another only at some intermediate step. We further observe that the translation performance highly depends on the chosen timestep for domain transfer, and therefore propose a practical strategy to automatically select an appropriate timestep for a given task. We evaluate our approach on a range of I2I applications, including image stylization, image colorization, segmentation to image, and sketch to image, to validate its efficacy and general utility. The comparisons show that our DMT surpasses existing methods in both quality and efficiency. Code will be made publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑