Despite the success achieved by existing image generation and editing methods, current models still struggle with complex problems including intricate text prompts, and the absence of verification and self-correction mechanisms makes the generated images unreliable. Meanwhile, a single model tends to specialize in particular tasks and possess the corresponding capabilities, making it inadequate for fulfilling all user requirements. We propose GenArtist, a unified image generation and editing system, coordinated by a multimodal large language model (MLLM) agent. We integrate a comprehensive range of existing models into the tool library and utilize the agent for tool selection and execution. For a complex problem, the MLLM agent decomposes it into simpler sub-problems and constructs a tree structure to systematically plan the procedure of generation, editing, and self-correction with step-by-step verification. By automatically generating missing position-related inputs and incorporating position information, the appropriate tool can be effectively employed to address each sub-problem. Experiments demonstrate that GenArtist can perform various generation and editing tasks, achieving state-of-the-art performance and surpassing existing models such as SDXL and DALL-E 3, as can be seen in Fig. 1. Project page is https://zhenyuw16.github.io/GenArtist_page.
Multimodal Image Synthesis and Editing: The Generative AI Era
Max Planck Institute for Informatics（马克斯·普朗克信息研究所）；S-Lab, Nanyang Technological University（南洋理工大学S实验室）；Nanyang Technological University（南洋理工大学）；Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editing
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
Towards Efficient Exemplar Based Image Editing with Multimodal VLMs
Indian Institute of Technology, Bombay（印度理工学院班加罗尔分校）；Indian Institute of Technology, Roorkee（印度理工学院罗尔基分校）；Microsoft Research（微软研究院）；Stanford University（斯坦福大学）；Adobe MDSR（Adobe MDSR实验室）；Carnegie Mellon University（卡内基梅隆大学）
SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models
University of Macau（澳门大学）；ARC Lab, Tencent PCG（腾讯平台与内容事业群ARC实验室）；Tencent AI Lab（腾讯人工智能实验室）；Shenzhen Institute of Advanced Technology（深圳先进技术研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Tsinghua University（清华大学）
Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing
Media Integration and Communication Center (MICC), University of Florence（佛罗伦萨大学媒体集成与传播中心（MICC））；University of Florence（佛罗伦萨大学）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）；University of Pisa（比萨大学）
DPDEdit: Detail-Preserved Diffusion Models for Multimodal Fashion Image Editing
Shenzhen Technology University（深圳科技大学）；University of Washington（华盛顿大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）
ReEdit: Multimodal Exemplar-Based Image Editing with Diffusion Models
Indian Institute of Technology, Roorkee（鲁尔基印度理工学院）；Adobe MDSR；Microsoft Research（微软研究院）；Indian Institute of Technology, Bombay（孟买印度理工学院）；Stanford University（斯坦福大学）；Carnegie Mellon University（卡内基梅隆大学）
Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow Matching
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Google Research（谷歌研究）；Sony Research, India（索尼印度研究）；Indian Institute of Technology, Roorkee（印度理工学院罗尔基分校）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
InterCoG: Towards Spatially Precise Image Editing with Interleaved Chain-of-Grounding Reasoning
Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算机学院）；Zhengzhou Advanced Research Institute of Harbin Institute of Technology（哈尔滨工业大学郑州先进研究院）；Huawei Noah’s Ark Lab（华为诺亚实验室）；Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology（深圳先进技术大学人工智能研究院）
Beyond Textual CoT: Interleaved Text-Image Chains with Deep Confidence Reasoning for Image Editing
Shanghai Jiaotong University（上海交通大学）；Huawei Noah’s Ark Lab（华为诺亚实验室）；East China Normal University（华东师范大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）
Factuality Matters: When Image Generation and Editing Meet Structured Visuals
CUHK MMLab（香港大学多模态实验室）；Beihang University（北京航空航天大学）；Krea AI（Krea人工智能）；Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Lab（上海人工智能实验室）；Hugging Face；National University of Singapore（新加坡国立大学）；ByteDance（字节跳动）；The University of Hong Kong（香港大学）
LMM4Edit: Benchmarking and Evaluating Multimodal Image Editing with LMMs
Institute of Image Communication and Network Engineering, Shanghai JiaoTong University（上海交通大学图像通信与网络工程研究所）；University of Electronic and Science Technology of China（电子科技大学）；Nanyang Technological University（南洋理工大学）
