TalkPhoto: 一种多功能的无需训练的对话式智能图像编辑助手
TalkPhoto: A Versatile Training-Free Conversational Assistant for Intelligent Image Editing
AI总结：
TalkPhoto通过对话交互实现无需训练的多功能图像编辑，利用提示模板分层调用现有高级编辑方法，提升编辑精度与质量。
摘要：
得益于大型语言模型（LLMs）强大的语言理解能力，现有的基于指令的图像编辑方法引入了多模态大型语言模型（MLLMs）以促进指令与图像之间的信息交换，确保图像编辑的可控性和灵活性。然而，这些框架往往需要构建多指令数据集来训练模型以处理多种编辑任务，这不仅耗时且劳动强度大，而且无法达到满意的效果。在本文中，我们提出了TalkPhoto，一种多功能的无需训练的图像编辑框架，通过对话交互实现精确的图像操作。我们通过专门设计的提示模板指导开源LLM，在接收到指令后分析用户需求，并分层调用现有的高级编辑方法，整个过程无需额外训练。此外，我们实现了插件式且高效的图像编辑方法调用，使复杂和未见过的编辑任务能够被整合到当前框架中，从而实现稳定且高质量的编辑结果。大量实验表明，我们的方法不仅能够以更少的token消耗提供更精确的调用，而且在各种图像编辑任务中都实现了更高的编辑质量。
英文摘要：
Thanks to the powerful language comprehension capabilities of Large Language Models (LLMs), existing instruction-based image editing methods have introduced Multimodal Large Language Models (MLLMs) to promote information exchange between instructions and images, ensuring the controllability and flexibility of image editing. However, these frameworks often build a multi-instruction dataset to train the model to handle multiple editing tasks, which is not only time-consuming and labor-intensive but also fails to achieve satisfactory results. In this paper, we present TalkPhoto, a versatile training-free image editing framework that facilitates precise image manipulation through conversational interaction. We instruct the open-source LLM with a specially designed prompt template to analyze user needs after receiving instructions and hierarchically invoke existing advanced editing methods, all without additional training. Moreover, we implement a plug-and-play and efficient invocation of image editing methods, allowing complex and unseen editing tasks to be integrated into the current framework, achieving stable and high-quality editing results. Extensive experiments demonstrate that our method not only provides more accurate invocation with fewer token consumption but also achieves higher editing quality across various image editing tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
