面向文本到图像扩散模型的闪电般快速的图像反演与编辑
Lightning-Fast Image Inversion and Editing for Text-to-Image Diffusion Models
AI总结：
本文提出引导式牛顿-拉夫森反演方法，将扩散反演形式化为求隐式方程根的问题，在0.4秒内实现高质量图像反演与编辑，支持交互式应用。
摘要：
扩散反演是指给定一幅图像和描述该图像的文本提示，寻找能够生成完全相同图像的噪声潜变量的问题。当前大多数确定性反演技术通过近似求解一个隐式方程来运作，可能收敛缓慢或产生较差的重建图像。我们将该问题形式化为寻找隐式方程的根，并开发了一种高效求解方法。我们的解决方案基于数值分析中著名的牛顿-拉夫森（Newton-Raphson, NR）方法。我们表明，直接应用NR在计算上是不可行的，而将其朴素地转化为计算上可行的替代方案往往收敛到分布外解，导致重建和编辑质量较差。因此，我们推导出一种高效的引导式公式，能够快速收敛并提供高质量的重建和编辑。我们在三个流行的开源扩散模型上展示了我们的方法在真实图像编辑中的效果：Stable Diffusion、SDXL-Turbo和Flux，并使用不同的确定性调度器。我们的解决方案Guided Newton-Raphson Inversion在少步模型（SDXL-Turbo和Flux.1）上可在0.4秒内（在A100 GPU上）完成图像反演，为交互式图像编辑打开了大门。我们进一步展示了在图像插值和稀有物体生成方面的改进结果。
英文摘要：
Diffusion inversion is the problem of taking an image and a text prompt that describes it and finding a noise latent that would generate the exact same image. Most current deterministic inversion techniques operate by approximately solving an implicit equation and may converge slowly or yield poor reconstructed images. We formulate the problem by finding the roots of an implicit equation and devlop a method to solve it efficiently. Our solution is based on Newton-Raphson (NR), a well-known technique in numerical analysis. We show that a vanilla application of NR is computationally infeasible while naively transforming it to a computationally tractable alternative tends to converge to out-of-distribution solutions, resulting in poor reconstruction and editing. We therefore derive an efficient guided formulation that fastly converges and provides high-quality reconstructions and editing. We showcase our method on real image editing with three popular open-sourced diffusion models: Stable Diffusion, SDXL-Turbo, and Flux with different deterministic schedulers. Our solution, Guided Newton-Raphson Inversion, inverts an image within 0.4 sec (on an A100 GPU) for few-step models (SDXL-Turbo and Flux.1), opening the door for interactive image editing. We further show improved results in image interpolation and generation of rare objects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
