什么是、是否以及如何？揭示图像推理中的过程奖励模型
What, Whether and How? Unveiling Process Reward Models for Thinking with Images Reasoning
AI总结：
本研究提出首个图像推理范式下的PRMs综合基准，定义7种细粒度错误类型，揭示当前LVLMs在视觉推理评估中的不足，为改进PRMs提供基础。
摘要：
大型视觉语言模型（LVLMs）的快速发展已证明了其在各种视觉任务中的出色能力。在此基础上，图像推理范式应运而生，使模型能够在每个推理步骤中动态编辑和重新编码视觉信息，模拟人类视觉处理过程。然而，这种范式引入了显著的挑战，因为在推理过程中可能会出现多种错误。这需要过程奖励模型（PRMs）来区分积极和消极的推理步骤，但现有的PRMs基准主要以文本为中心，缺乏在该范式下的全面评估。为了解决这些缺口，本工作引入了第一个专门针对图像推理范式评估PRMs的综合基准。我们的主要贡献是：（1）通过广泛分析推理轨迹和引导搜索实验，我们定义了7种细粒度的错误类型，并展示了对专用PRMs的必要性和改进潜力。（2）我们构建了一个包含1,206个手动标注的图像推理轨迹的综合基准，涵盖4个类别和16个子类别，以实现对PRMs的细粒度评估。（3）我们的实验分析表明，当前的LVLMs在作为有效PRMs方面存在不足，表现出在视觉推理过程评估中的有限能力，且在不同错误类型、积极评估偏见和推理步骤位置的敏感性上存在显著差异。这些发现证明了我们基准的有效性，并为推进LVLMs中的PRMs奠定了关键基础。
英文摘要：
The rapid advancement of Large Vision Language Models (LVLMs) has demonstrated excellent abilities in various visual tasks. Building upon these developments, the thinking with images paradigm has emerged, enabling models to dynamically edit and re-encode visual information at each reasoning step, mirroring human visual processing. However, this paradigm introduces significant challenges as diverse errors may occur during reasoning processes. This necessitates Process Reward Models (PRMs) for distinguishing positive and negative reasoning steps, yet existing benchmarks for PRMs are predominantly text-centric and lack comprehensive assessment under this paradigm. To address these gaps, this work introduces the first comprehensive benchmark specifically designed for evaluating PRMs under the thinking with images paradigm. Our main contributions are: (1) Through extensive analysis of reasoning trajectories and guided search experiments with PRMs, we define 7 fine-grained error types and demonstrate both the necessity for specialized PRMs and the potential for improvement. (2) We construct a comprehensive benchmark comprising 1,206 manually annotated thinking with images reasoning trajectories spanning 4 categories and 16 subcategories for fine-grained evaluation of PRMs. (3) Our experimental analysis reveals that current LVLMs fall short as effective PRMs, exhibiting limited capabilities in visual reasoning process evaluation with significant performance disparities across error types, positive evaluation bias, and sensitivity to reasoning step positions. These findings demonstrate the effectiveness of our benchmark and establish crucial foundations for advancing PRMs in LVLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
