EditScore: 通过高保真奖励建模解锁图像编辑的在线强化学习
EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling
AI总结：
EditScore通过高保真奖励建模解锁图像编辑的在线强化学习，提供从基准到奖励建模到RL训练的系统路径。
摘要：
指导指令的图像编辑已经取得了显著进展，但当前模型仍然面临复杂指令的挑战，通常需要多个样本才能产生期望的结果。强化学习（RL）提供了一个有希望的解决方案，但其在图像编辑中的应用受到缺乏高保真、高效的奖励信号的严重阻碍。在本工作中，我们提出了一种全面的方法来克服这一障碍，核心是开发一个最先进的专用奖励模型。我们首先介绍了EditReward-Bench，一个全面的基准，用于系统评估奖励模型的编辑质量。基于这一基准，我们开发了EditScore，一系列奖励模型（7B-72B）用于评估指导指令的图像编辑质量。通过精心的数据整理和过滤，EditScore有效匹配了学习专有VLMs的性能。此外，结合一种针对EditScore生成性质的有效自集成策略，我们的最大变体甚至在基准中超过了GPT-5。我们随后表明，高保真奖励模型是解锁图像编辑在线RL的关键。我们的实验显示，尽管即使最大的开源VLMs无法提供有效的学习信号，EditScore能够实现高效且稳健的策略优化。将我们的框架应用于强大的基础模型OmniGen2，最终模型显示出显著且一致的性能提升。总体而言，这项工作提供了从基准到奖励建模到RL训练在图像编辑中的系统路径，表明高保真、领域专用的奖励模型是解锁该领域RL潜力的关键。
英文摘要：
Instruction-guided image editing has achieved remarkable progress, yet current models still face challenges with complex instructions and often require multiple samples to produce a desired result. Reinforcement Learning (RL) offers a promising solution, but its adoption in image editing has been severely hindered by the lack of a high-fidelity, efficient reward signal. In this work, we present a comprehensive methodology to overcome this barrier, centered on the development of a state-of-the-art, specialized reward model. We first introduce EditReward-Bench, a comprehensive benchmark to systematically evaluate reward models on editing quality. Building on this benchmark, we develop EditScore, a series of reward models (7B-72B) for evaluating the quality of instruction-guided image editing. Through meticulous data curation and filtering, EditScore effectively matches the performance of learning proprietary VLMs. Furthermore, coupled with an effective self-ensemble strategy tailored for the generative nature of EditScore, our largest variant even surpasses GPT-5 in the benchmark. We then demonstrate that a high-fidelity reward model is the key to unlocking online RL for image editing. Our experiments show that, while even the largest open-source VLMs fail to provide an effective learning signal, EditScore enables efficient and robust policy optimization. Applying our framework to a strong base model, OmniGen2, results in a final model that shows a substantial and consistent performance uplift. Overall, this work provides the first systematic path from benchmarking to reward modeling to RL training in image editing, showing that a high-fidelity, domain-specialized reward model is the key to unlocking the full potential of RL in this domain.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
