DragLoRA: 面向扩散模型中基于拖拽的图像编辑的LoRA适配器在线优化
DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
AI总结：
针对传统拖拽式图像编辑精度低、效率差的问题，提出集成LoRA适配器的DragLoRA框架，通过去噪分数蒸馏损失和自适应优化方案，显著提升编辑的控制精度与计算效率。
摘要：
预训练扩散模型中的基于拖拽的编辑为操控前景对象提供了一种精确且灵活的方式。传统方法直接优化从DDIM反演得到的输入特征，迭代调整这些特征以引导控制点向目标位置移动。然而，这类方法往往因运动监督中特征的表征能力有限而存在精度不足的问题，同时点跟踪所需的大搜索空间也会导致效率低下。\n为解决这些局限，我们提出了DragLoRA——一种将LoRA（低秩适配）适配器集成到基于拖拽的编辑流程中的新型框架。为增强LoRA适配器的训练效果，我们引入了额外的去噪分数蒸馏损失，通过使在线模型的输出与原始模型的输出对齐来对其进行正则化。此外，我们通过使用更新后的LoRA调整输入特征，提升了运动监督的一致性，为后续操作提供了更稳定、更准确的输入特征。\n在此基础上，我们设计了一种自适应优化方案，可在两种模式间动态切换，在不损失精度的前提下优先保障效率。大量实验表明，DragLoRA显著提升了基于拖拽的图像编辑的控制精度和计算效率。DragLoRA的代码可在以下链接获取：https://github.com/Sylvie-X/DragLoRA。
英文摘要：
Drag-based editing within pretrained diffusion model provides a precise and flexible way to manipulate foreground objects. Traditional methods optimize the input feature obtained from DDIM inversion directly, adjusting them iteratively to guide handle points towards target locations. However, these approaches often suffer from limited accuracy due to the low representation ability of the feature in motion supervision, as well as inefficiencies caused by the large search space required for point tracking. To address these limitations, we present DragLoRA, a novel framework that integrates LoRA (Low-Rank Adaptation) adapters into the drag-based editing pipeline. To enhance the training of LoRA adapters, we introduce an additional denoising score distillation loss which regularizes the online model by aligning its output with that of the original model. Additionally, we improve the consistency of motion supervision by adapting the input features using the updated LoRA, giving a more stable and accurate input feature for subsequent operations. Building on this, we design an adaptive optimization scheme that dynamically toggles between two modes, prioritizing efficiency without compromising precision. Extensive experiments demonstrate that DragLoRA significantly enhances the control precision and computational efficiency for drag-based image editing. The Codes of DragLoRA are available at: https://github.com/Sylvie-X/DragLoRA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
