FastEdit：基于语义感知扩散微调的快速文本引导单图像编辑
FastEdit: Fast Text-Guided Single-Image Editing via Semantic-Aware Diffusion Fine-Tuning
AI总结：
FastEdit通过语义感知扩散微调、特征空间条件图像到图像模型及LoRA技术，将文本引导单图像编辑从7分钟加速至17秒，同时保持可比编辑效果。
摘要：
传统的文本引导单图像编辑方法需要两阶段流程，包括对目标文本嵌入进行超过1K次迭代的微调，以及对生成模型再进行1.5K次迭代的微调。尽管这能确保生成图像与输入图像和目标文本高度一致，但该流程通常每张图像需要7分钟，其耗时特性对实际应用构成挑战。为解决这一瓶颈，我们提出FastEdit，一种具有语义感知扩散微调的快速文本引导单图像编辑方法，将编辑过程大幅加速至仅需17秒。FastEdit简化了生成模型的微调阶段，将其从1.5K次迭代减少到仅50次迭代。对于扩散微调，我们根据输入图像与目标文本之间的语义差异选取特定的时间步值。此外，FastEdit通过使用以特征空间而非文本嵌入空间为条件的图像到图像模型，绕过了初始微调步骤。它能够在同一特征空间内有效对齐目标文本提示和输入图像，并节省大量处理时间。另外，我们将参数高效微调技术LoRA应用于U-net。借助LoRA，FastEdit将模型的可训练参数最小化至原始规模的仅0.37%。同时，我们能够在显著降低计算开销的情况下取得可比的编辑效果。我们进行了大量实验来验证所提方法的编辑性能，并展示了有前景的编辑能力，包括内容添加、风格迁移、背景替换和姿态操控等。
英文摘要：
Conventional Text-guided single-image editing approaches require a two-step process, including fine-tuning the target text embedding for over 1K iterations and the generative model for another 1.5K iterations. Although it ensures that the resulting image closely aligns with both the input image and the target text, this process often requires 7 minutes per image, posing a challenge for practical application due to its time-intensive nature. To address this bottleneck, we introduce FastEdit, a fast text-guided single-image editing method with semantic-aware diffusion fine-tuning, dramatically accelerating the editing process to only 17 seconds. FastEdit streamlines the generative model's fine-tuning phase, reducing it from 1.5K to a mere 50 iterations. For diffusion fine-tuning, we adopt certain time step values based on the semantic discrepancy between the input image and target text. Furthermore, FastEdit circumvents the initial fine-tuning step by utilizing an image-to-image model that conditions on the feature space, rather than the text embedding space. It can effectively align the target text prompt and input image within the same feature space and save substantial processing time. Additionally, we apply the parameter-efficient fine-tuning technique LoRA to U-net. With LoRA, FastEdit minimizes the model's trainable parameters to only 0.37\% of the original size. At the same time, we can achieve comparable editing outcomes with significantly reduced computational overhead. We conduct extensive experiments to validate the editing performance of our approach and show promising editing capabilities, including content addition, style transfer, background replacement, and posture manipulation, etc.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
