Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis
University of Virginia（弗吉尼亚大学）；Columbia University（哥伦比亚大学）；Vanderbilt University（范德比大学）；Adobe Research（Adobe研究）；Dolby Laboratories（杜比实验室）；Cisco Research（思科研究）；University of Southern California（南加州大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of Oregon（俄勒冈大学）；Texas A&M University（德克萨斯大学）
Runzhou Liu, Hailey Weingord, Sejal Mittal, Prakhar Dungarwal, Anusha Nandula, Bo Ni, Samyadeep Basu, Hongjie Chen, Nesreen K. Ahmed, Li Li, Jiayi Zhang, Kousta… 展开作者
Runzhou Liu, Hailey Weingord, Sejal Mittal, Prakhar Dungarwal, Anusha Nandula, Bo Ni, Samyadeep Basu, Hongjie Chen, Nesreen K. Ahmed, Li Li, Jiayi Zhang, Koustava Goswami, Subhojyoti Mukherjee, Branislav Kveton, Puneet Mathur, Franck Dernoncourt, Yue Zhao, Yu Wang, Ryan A. Rossi, Zhengzhong Tu, Hongru Du
Evaluating image editing models remains challenging due to the coarse granularity and limited interpretability of traditional metrics, which often fail to capture aspects important to human perception and intent. Such metrics frequently reward visually plausible outputs while overlooking controllability, edit localization, and faithfulness to user instructions. In this work, we introduce a fine-grained Multimodal Large Language Model (MLLM)-as-a-Judge framework for image editing that decomposes common evaluation notions into twelve fine-grained interpretable factors spanning image preservation, edit quality, and instruction fidelity. Building on this formulation, we present a new human-validated benchmark that integrates human judgments, MLLM-based evaluations, model outputs, and traditional metrics across diverse image editing tasks. Through extensive human studies, we show that the proposed MLLM judges align closely with human evaluations at a fine granularity, supporting their use as reliable and scalable evaluators. We further demonstrate that traditional image editing metrics are often poor proxies for these factors, failing to distinguish over-edited or semantically imprecise outputs, whereas our judges provide more intuitive and informative assessments in both offline and online settings. Together, this work introduces a benchmark, a principled factorization, and empirical evidence positioning fine-grained MLLM judges as a practical foundation for studying, comparing, and improving image editing approaches.
ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs' Capability via Chart Editing
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Harbin Institute of Technology（哈尔滨工业大学）；Tianjin University（天津大学）；Tsinghua University（清华大学）
Jury Duty: Calibration and Orientation Failures in MLLM-as-a-Judge Under Cultural Ambiguity
Salesforce AI Research（Salesforce AI 研究院）；University of Cambridge（剑桥大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Carnegie Mellon University（卡内基梅隆大学）；UIUC（伊利诺伊大学厄巴纳-香槟分校）
Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs
School of Information Science and Engineering, Yunnan University（云南大学信息科学与工程学院）；School of Software, Yunnan University（云南大学软件学院）；National University of Singapore（新加坡国立大学）；School of Engineering, Yunnan University（云南大学工程学院）
Judge Before Answer: Can MLLM Discern the False Premise in Question?
School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；Inner Mongolia Research Institute, Shanghai Jiao Tong University（上海交通大学内蒙古研究院）
SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models
University of Macau（澳门大学）；ARC Lab, Tencent PCG（腾讯平台与内容事业群ARC实验室）；Tencent AI Lab（腾讯人工智能实验室）；Shenzhen Institute of Advanced Technology（深圳先进技术研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Tsinghua University（清华大学）
Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning
Tencent Youtu Lab（腾讯云图实验室）；Sichuan University（四川大学）；University of the Chinese Academy of Sciences（中国科学院大学）；Fudan University（复旦大学）；Zhejiang University（浙江大学）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）
East China Normal University（华东师范大学）；Xiaohongshu Inc.（小红书）；KLATASDS, MOE, China（中国教育部统计与数据科学重点实验室）；The Chinese University of Hong Kong（香港中文大学）；Zhejiang University（浙江大学）；Fudan University（复旦大学）；University of Oxford（牛津大学）；Shanghai Jiao Tong University（上海交通大学）；University of Science and Technology of China（中国科学技术大学）；Nanjing University（南京大学）
Making Image Editing Easier via Adaptive Task Reformulation with Agentic Executions
Nanjing University（南京大学）；Beijing Institute of Technology（北京理工大学）；College of Artificial Intelligence, China University of Petroleum (Beijing)（中国石油大学（北京）人工智能学院）；Beijing University of Posts and Telecommunications（北京邮电大学）；Fudan University（复旦大学）
