利用 LLM 实现即时指令引导的图像编辑
Leveraging LLMs for On-the-Fly Instruction Guided Image Editing
AI总结：
针对仅依据自然语言指令编辑图像的任务，本文提出一种无需训练或微调的即时方法，通过图像描述、DDIM 反演、编辑方向嵌入和图像编辑三步流程，在 MAGICBRIFF 数据集上超越现有最先进模型。
摘要：
鉴于近期利用语言处理与图像处理两大研究领域综合优势所取得的令人瞩目的进展，二者的结合持续吸引着越来越多的关注。在这些进展中，仅依据自然语言指令编辑图像的任务是一项极具挑战性的工作。尽管近期针对该任务的方法都以这种或那种方式依赖某种形式的前期准备、训练或微调，本文探索了一种新颖方法：我们提出了一种无需准备的方法，可实现即时的指令引导图像编辑。该方法由三个妥善协调的步骤组成：借助图像描述和 DDIM 反演，随后获取编辑方向嵌入，最后进行真正的图像编辑。在无需前期准备的同时，我们的方法被证明有效且具有竞争力，在 MAGICBRUSH 数据集上评估时，其表现优于近期用于该任务的最先进模型。
英文摘要：
The combination of language processing and image processing keeps attracting increased interest given recent impressive advances that leverage the combined strengths of both domains of research. Among these advances, the task of editing an image on the basis solely of a natural language instruction stands out as a most challenging endeavour. While recent approaches for this task resort, in one way or other, to some form of preliminary preparation, training or fine-tuning, this paper explores a novel approach: We propose a preparation-free method that permits instruction-guided image editing on the fly. This approach is organized along three steps properly orchestrated that resort to image captioning and DDIM inversion, followed by obtaining the edit direction embedding, followed by image editing proper. While dispensing with preliminary preparation, our approach demonstrates to be effective and competitive, outperforming recent, state of the art models for this task when evaluated on the MAGICBRUSH dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
