多模态奖励基准2：评估多模态奖励模型用于交错文本和图像
Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image
AI总结：
多模态奖励基准2评估多模态奖励模型在交错文本和图像任务中的性能，通过专家标注数据和先进模型对比，揭示了不同模型在多模态理解与生成任务中的准确率差异。
摘要：
奖励模型（RMs）对于训练大语言模型（LLMs）至关重要，但在处理交错图像和文本序列的多模态模型中仍处于探索初期。我们介绍了多模态奖励基准2（MMRB2），这是首个全面评估奖励模型在多模态理解和（交错）生成任务上的基准。MMRB2涵盖四个任务：文本到图像、图像编辑、交错生成和多模态推理（“带图像的思考”），每个任务提供来自23个模型和代理的1000对专家标注偏好数据，涵盖21个源任务。MMRB2设计包含：（1）实用但具有挑战性的提示；（2）来自最先进模型和代理的响应；以及（3）具有强人类专家共识的偏好对，通过集合过滤策略进行筛选。使用MMRB2，我们研究了每个子任务的现有判断者，包括多模态LLM-as-a-judge和通过人类偏好训练的模型。最新的Gemini 3 Pro达到75-80%的准确率。GPT-5和Gemini 2.5 Pro达到66-75%的准确率，与人类的>90%相比，但超过了广泛使用的GPT-4o（59%）。表现最佳的开源模型Qwen3-VL-32B达到与Gemini 2.5 Flash（64%）相似的准确率。我们还展示了MMRB2的表现与下游任务成功之间存在强相关性，使用Best-of-N采样进行深入分析，并展示了奖励模型未来改进的关键领域。
英文摘要：
Reward models (RMs) are essential for training large language models (LLMs), but remain underexplored for omni models that handle interleaved image and text sequences. We introduce Multimodal RewardBench 2 (MMRB2), the first comprehensive benchmark for reward models on multimodal understanding and (interleaved) generation. MMRB2 spans four tasks: text-to-image, image editing, interleaved generation, and multimodal reasoning ("thinking-with-images"), providing 1,000 expert-annotated preference pairs per task from 23 models and agents across 21 source tasks. MMRB2 is designed with: (1) practical but challenging prompts; (2) responses from state-of-the-art models and agents; and (3) preference pairs with strong human-expert consensus, curated via an ensemble filtering strategy. Using MMRB2, we study existing judges for each subtask, including multimodal LLM-as-a-judge and models trained with human preferences. The latest Gemini 3 Pro attains 75-80% accuracy. GPT-5 and Gemini 2.5 Pro reach 66-75% accuracy, compared to >90% for humans, yet surpass the widely used GPT-4o (59%). The best performing open-source model Qwen3-VL-32B achieves similar accuracies as Gemini 2.5 Flash (64%). We also show that MMRB2 performance strongly correlates with downstream task success using Best-of-N sampling and conduct an in-depth analysis that shows key areas to improve the reward models going forward.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
