arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.06136cs.CVcs.MM
收藏

用于超高分辨率图像编辑的免调优潜在扩散模型

Tuning-Free Latent Diffusion Models for Ultrahigh-Resolution Image Editing

College of Computer Science and Artificial Intelligence, Wenzhou University(温州大学计算机科学与人工智能学院);Department of Computer Science, Memorial University of Newfoundland(纽芬兰纪念大学计算机科学系);AI Analytics Team, Nasdaq(纳斯达克人工智能分析团队);State Key Laboratory of CAD&CG, Zhejiang University(浙江大学CAD&CG国家重点实验室)

Wanglong Lu, Lingming Su, Kaijie Shi, Minglun Gong, Xiaogang Jin, Hanli Zhao, Xianta Jiang

AI总结:

针对现有图像编辑方法在高分辨率处理上的局限,提出免调优的UltraDiffEdit框架,通过多尺度渐进编辑、多补丁编码、全局-局部一致性去噪及补丁混合采样等技术,实现高质量超高分辨率图像编辑,处理能力达8K且灵活性高。

摘要:

近期基于扩散的生成模型在图像生成和编辑方面表现出色。但因内存限制和高分辨率训练图像收集成本高,现有方法通常限于线性分辨率低于1K的输入。现代移动设备拍摄的照片常达8K线性分辨率,当前能力与实际需求有差距。本文介绍了UltraDiffEdit,一种将现成潜在扩散模型扩展到超高分辨率的免调优图像编辑框架。它采用多尺度渐进编辑策略,运用多补丁编码和全局-局部一致性去噪技术,还引入基于补丁的混合采样方法。实验表明其编辑质量和灵活性优越,仅用单个NVIDIA GeForce RTX 3090 GPU就能处理高达8K的图像分辨率,且代码开源。

英文摘要:

Recent diffusion-based generative models have shown impressive performance in image generation and editing. However, due to memory limitations and the high cost of collecting high-resolution training images, existing methods are typically restricted to inputs with linear resolutions below 1K. In contrast, photos captured by modern mobile devices often reach linear resolutions up to 8K, revealing a significant gap between current capabilities and real-world demands. Simply upscaling low-resolution edited results often results in visually enlarged but blurry images that lack fine details. This paper introduces UltraDiffEdit, a novel, tuning-free image editing framework that extends off-the-shelf latent diffusion models (LDMs) to ultrahigh resolutions. UltraDiffEdit employs a multi-scale progressive editing strategy, iteratively blending high-resolution edited content with unedited areas in a coarse-to-fine manner. We employ multi-patch encoding to preserve both edited and unedited visual details within the latent space. To mitigate editing artifacts, our global-local consistency denoising technique consistently integrates edited and unedited latent features, ensuring smooth transition at editing boundaries from the latent representation to the final image. We also introduce a patch-based hybrid sampling approach that captures local, intermediate, and global features, ensuring semantic coherence and enhancing fine detail during denoising. We conduct extensive experiments demonstrating UltraDiffEdit's superior editing quality and flexibility: it can handle image resolutions up to 8K using only a single NVIDIA GeForce RTX 3090 GPU. The source code is publicly available at https://github.com/LonglongaaaGo/UltraDiffEdit.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑