ThinkDrag:基于语义拖拽与视觉推理的图像编辑
ThinkDrag: Semantic Drag-Based Image Editing with Visual Reasoning
AI总结:
ThinkDrag是具备视觉推理能力的多模态图像编辑框架,通过关联点约束与语义变换、生成推理轨迹提升编辑效果,在DragBench++基准上实现了最先进性能。
摘要:
基于拖拽的图像编辑通过允许用户指定控制点和目标点,提供了一种用于空间可控图像操作的直观界面。现有方法通过基于优化、单步及前馈的公式取得了显著进展,但它们常将拖拽指令解释为几何位移,这在期望的编辑依赖于解释图像内容并从稀疏点约束中推断合理语义变换时,会限制其有效性。我们提出ThinkDrag,这是一个用于基于拖拽的图像编辑的统一多模态框架,具备视觉推理能力。ThinkDrag被训练为将点约束与有意义的对象级和场景级变换相关联,还可选择性生成明确的推理轨迹,在生成图像前解释预期的编辑内容,该推理轨迹提升了可解释性,尤其适用于模糊拖拽场景。受跨图像注意力分析的启发,拖拽端点标记提供了明确的对应信号,将每个拖拽锚定在图像空间中。为支持该框架,我们构建了带有推理轨迹的语义拖拽变换监督数据集,并引入DragBench++,这是一个针对具有参考编辑解决方案的挑战性基于拖拽的编辑场景的基准。实验表明,ThinkDrag实现了最先进的性能,在保持有竞争力的点跟随精度的同时,提升了生成质量和合理性。
英文摘要:
Drag-based image editing provides an intuitive interface for spatially controlled image manipulation by allowing users to specify handle and target points. Existing methods have made substantial progress through optimization-based, one-step, and feed-forward formulations, but they often interpret drag instructions as geometric displacements. This limits their effectiveness when the desired edit depends on interpreting image content and inferring a plausible semantic transformation from sparse point constraints. We introduce ThinkDrag, a unified multimodal framework for drag-based image editing with visual reasoning. ThinkDrag is trained to associate point constraints with meaningful object- and scene-level transformations, and can optionally produce an explicit reasoning trace that interprets the intended edit before image generation. This reasoning trace improves interpretability, particularly for ambiguous drags. Drag Endpoint Markers, motivated by an analysis of cross-image attention, provide an explicit correspondence signal that grounds each drag in image space. To support this framework, we construct a supervised dataset of semantic drag transformations paired with reasoning traces and introduce DragBench++, a benchmark targeting challenging drag-based editing scenarios with reference edit solutions. Experiments show that ThinkDrag achieves state-of-the-art performance, improving generation quality and plausibility while maintaining competitive point-following precision.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。