Edit-Compass 与 EditReward-Compass：图像编辑与奖励建模的统一基准
Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling
AI总结：
本文提出Edit-Compass和EditReward-Compass，通过精细多维评估框架和真实奖励建模场景，解决现有图像编辑和奖励模型评估中的不足。
摘要：
近期图像编辑模型在指令跟随、多模态理解和复杂视觉编辑方面取得了显著进展。然而，现有基准往往无法真实反映人类判断，尤其是对于强前沿模型，由于任务难度有限和粗粒度评估协议。同时，奖励模型在基于强化学习的图像编辑优化中变得越来越重要，但现有奖励模型基准仍依赖不现实的评估设置，偏离实际RL场景。这些限制阻碍了图像编辑模型和奖励模型的可靠评估。为解决这些挑战，我们引入Edit-Compass和EditReward-Compass，一个统一的评估套件。Edit-Compass包含2,388个精心标注的实例，涵盖六个逐步挑战性的任务类别，包括世界知识推理、视觉推理和多图像编辑。除了广泛的任务覆盖，Edit-Compass采用基于结构化推理和精心设计评分标准的细粒度多维评估框架。同时，EditReward-Compass包含2,251个偏好对，模拟RL优化期间的真实奖励建模场景。
英文摘要：
Recent image editing models have achieved remarkable progress in instruction following, multimodal understanding, and complex visual editing. However, existing benchmarks often fail to faithfully reflect human judgment, especially for strong frontier models, due to limited task difficulty and coarse-grained evaluation protocols. In parallel, reward models have become increasingly important for RL-based image editing optimization, yet existing reward model benchmarks still rely on unrealistic evaluation settings that deviate from practical RL scenarios. These limitations hinder reliable assessment of both image editing models and reward models. To address these challenges, we introduce Edit-Compass and EditReward-Compass, a unified evaluation suite for image editing and reward modeling. Edit-Compass contains 2,388 carefully annotated instances spanning six progressively challenging task categories, covering capabilities such as world knowledge reasoning, visual reasoning, and multi-image editing. Beyond broad task coverage, Edit-Compass adopts a fine-grained multidimensional evaluation framework based on structured reasoning and carefully designed scoring rubrics. In parallel, EditReward-Compass contains 2,251 preference pairs that simulate realistic reward modeling scenarios during RL optimization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
