ClickDiffusion：利用 LLM 实现交互式精确图像编辑
ClickDiffusion: Harnessing LLMs for Interactive Precise Image Editing
AI总结：
本文提出 ClickDiffusion，通过将图像与多模态指令序列化为文本表示，结合 LLM 和用户点击式视觉反馈，实现精确的交互式图像布局与外观编辑。
摘要：
近来，研究人员提出了多种强大的系统，可利用自然语言指令生成和操控图像。然而，仅凭文本很难精确指定许多常见类别的图像变换。例如，用户可能希望在一张包含多只相似狗的图像中，改变其中某一只特定狗的位置和品种。仅靠自然语言完成这项任务相当困难，用户需要费力编写复杂提示词，既要消除目标狗的歧义，又要描述目标位置。我们提出 ClickDiffusion，这是一个用于精确图像操控和生成的系统，将自然语言指令与用户通过直接操控界面提供的视觉反馈相结合。我们证明，通过将图像和多模态指令同时序列化为文本表示，可以利用 LLM 对图像的布局和外观执行精确变换。代码可在 https://github.com/poloclub/ClickDiffusion 获取。
英文摘要：
Recently, researchers have proposed powerful systems for generating and manipulating images using natural language instructions. However, it is difficult to precisely specify many common classes of image transformations with text alone. For example, a user may wish to change the location and breed of a particular dog in an image with several similar dogs. This task is quite difficult with natural language alone, and would require a user to write a laboriously complex prompt that both disambiguates the target dog and describes the destination. We propose ClickDiffusion, a system for precise image manipulation and generation that combines natural language instructions with visual feedback provided by the user through a direct manipulation interface. We demonstrate that by serializing both an image and a multi-modal instruction into a textual representation it is possible to leverage LLMs to perform precise transformations of the layout and appearance of an image. Code available at https://github.com/poloclub/ClickDiffusion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
