Inpaint4Drag:通过双向翘曲重利用图像修复模型实现基于拖拽的图像编辑
Inpaint4Drag: Repurposing Inpainting Models for Drag-Based Image Editing via Bidirectional Warping
AI总结:
针对现有拖拽式图像编辑方法精度低、速度慢等问题,提出Inpaint4Drag框架,将编辑拆解为双向翘曲与图像修复,可适配任意修复模型,实现实时高精度的拖拽编辑效果。
摘要:
基于拖拽的图像编辑已成为直观操控图像的强大范式。然而,现有方法主要依赖对生成模型的隐空间进行操作,导致精度有限、反馈延迟且存在模型特定约束。为此,我们提出Inpaint4Drag——一种将拖拽式编辑拆解为像素空间双向翘曲与图像修复的新型框架。受物理世界中弹性物体形变的启发,我们将图像区域视为可形变材料,在用户操控下能保持自然形态。我们的方法在512×512分辨率下实现了实时翘曲预览(0.01秒)与高效修复(0.3秒),相比每次编辑需耗时数分钟的现有方法,显著提升了交互体验。通过将拖拽输入直接转换为标准图像修复格式,我们的方法可作为任意图像修复模型的通用适配器,无需修改架构,且能自动继承图像修复技术未来的所有改进。大量实验表明,该方法在保持实时性能的同时,实现了更优的视觉质量与精准控制。项目主页:https://visual-ai.github.io/inpaint4drag/
英文摘要:
Drag-based image editing has emerged as a powerful paradigm for intuitive image manipulation. However, existing approaches predominantly rely on manipulating the latent space of generative models, leading to limited precision, delayed feedback, and model-specific constraints. Accordingly, we present Inpaint4Drag, a novel framework that decomposes drag-based editing into pixel-space bidirectional warping and image inpainting. Inspired by elastic object deformation in the physical world, we treat image regions as deformable materials that maintain natural shape under user manipulation. Our method achieves real-time warping previews (0.01s) and efficient inpainting (0.3s) at 512x512 resolution, significantly improving the interaction experience compared to existing methods that require minutes per edit. By transforming drag inputs directly into standard inpainting formats, our approach serves as a universal adapter for any inpainting model without architecture modification, automatically inheriting all future improvements in inpainting technology. Extensive experiments demonstrate that our method achieves superior visual quality and precise control while maintaining real-time performance. Project page: https://visual-ai.github.io/inpaint4drag/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。