InstantDrag：提升基于拖拽的图像编辑中的交互性
InstantDrag: Improving Interactivity in Drag-based Image Editing
AI总结：
本文提出无需优化的 InstantDrag 流程，仅用图像和拖拽指令，通过 FlowGen 与 FlowDiffusion 分解运动及图像生成，实现快速、照片级真实且无需 mask 或文本提示的交互式拖拽编辑。
摘要：
基于拖拽的图像编辑近来因其交互性和精确性而受到欢迎。然而，尽管 text-to-image 模型能够在一秒内生成样本，拖拽编辑仍然滞后，原因在于难以在保持图像内容的同时准确反映用户交互。一些现有方法依赖计算密集的逐图像优化或复杂的基于引导的方法，需要额外输入，例如可移动区域的 mask 和文本提示，从而损害了编辑过程的交互性。我们提出 InstantDrag，这是一个无需优化的流程，可提升交互性和速度，仅需一张图像和一条拖拽指令作为输入。InstantDrag 由两个精心设计的网络组成：一个拖拽条件光流生成器（FlowGen）和一个光流条件扩散模型（FlowDiffusion）。通过将任务分解为运动生成和运动条件图像生成，InstantDrag 在真实世界视频数据集中学习基于拖拽的图像编辑的运动动态。我们通过在面部视频数据集和通用场景上的实验，证明 InstantDrag 能够在没有 mask 或文本提示的情况下执行快速、照片级真实的编辑。这些结果凸显了我们的方法在处理基于拖拽的图像编辑时的效率，使其成为交互式实时应用的一种有前景的解决方案。
英文摘要：
Drag-based image editing has recently gained popularity for its interactivity and precision. However, despite the ability of text-to-image models to generate samples within a second, drag editing still lags behind due to the challenge of accurately reflecting user interaction while maintaining image content. Some existing approaches rely on computationally intensive per-image optimization or intricate guidance-based methods, requiring additional inputs such as masks for movable regions and text prompts, thereby compromising the interactivity of the editing process. We introduce InstantDrag, an optimization-free pipeline that enhances interactivity and speed, requiring only an image and a drag instruction as input. InstantDrag consists of two carefully designed networks: a drag-conditioned optical flow generator (FlowGen) and an optical flow-conditioned diffusion model (FlowDiffusion). InstantDrag learns motion dynamics for drag-based image editing in real-world video datasets by decomposing the task into motion generation and motion-conditioned image generation. We demonstrate InstantDrag's capability to perform fast, photo-realistic edits without masks or text prompts through experiments on facial video datasets and general scenes. These results highlight the efficiency of our approach in handling drag-based image editing, making it a promising solution for interactive, real-time applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
