arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.11734cs.CVcs.AI

InverseMeetInsert:通过引导扩散模型中的几何累积反演实现鲁棒的真实图像编辑

InverseMeetInsert: Robust Real Image Editing via Geometric Accumulation Inversion in Guided Diffusion Models

UT Austin(德克萨斯大学奥斯汀分校)

Yan Zheng, Lemeng Wu

AI总结:

本研究提出无需训练的图像编辑技术GEO,通过几何累积损失优化DDIM反演并结合增强型图像提示技术,基于Stable Diffusion模型实现了真实图像的高保真定制化编辑。

摘要:

本文提出了Geometry-Inverse-Meet-Pixel-Insert(简称GEO),这是一种通用性极强的图像编辑技术,可满足用户在局部和全局尺度的定制化需求。该方法无缝融合文本提示与图像提示,生成多样且精准的编辑效果。值得注意的是,本方法无需训练,核心在于两项关键贡献:一是提出新型几何累积损失,优化DDIM反演以忠实保留像素空间的几何结构与布局;二是创新的增强型图像提示技术,将纯文本反演的像素级编辑与标准无分类器反演的潜空间几何引导相结合。基于公开的Stable Diffusion模型,我们在多种图像类型及高难度提示编辑场景下开展了大量评估,该方法始终能为真实图像生成高保真的编辑结果。

英文摘要:

In this paper, we introduce Geometry-Inverse-Meet-Pixel-Insert, short for GEO, an exceptionally versatile image editing technique designed to cater to customized user requirements at both local and global scales. Our approach seamlessly integrates text prompts and image prompts to yield diverse and precise editing outcomes. Notably, our method operates without the need for training and is driven by two key contributions: (i) a novel geometric accumulation loss that enhances DDIM inversion to faithfully preserve pixel space geometry and layout, and (ii) an innovative boosted image prompt technique that combines pixel-level editing for text-only inversion with latent space geometry guidance for standard classifier-free reversion. Leveraging the publicly available Stable Diffusion model, our approach undergoes extensive evaluation across various image types and challenging prompt editing scenarios, consistently delivering high-fidelity editing results for real images.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑