arXiv 2512.062762026-07-29cs.CVcs.AI

RefBench-PRO：面向感知与推理的指称表达理解基准

RefBench-PRO: Perceptual and Reasoning Oriented Benchmark for Referring Expression Comprehension

National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（国家人机混合增强智能重点实验室）；National Engineering Research Center for Visual Information and Applications（国家视觉信息与应用工程技术研究中心）；Institute of Artificial Intelligence and Robotics（人工智能与机器人研究院）；Xi’an Jiaotong University（西安交通大学）；Institute of Artificial Intelligence (TeleAI)（人工智能研究院（TeleAI））；China Telecom（中国电信）；Shanghai Jiao Tong University（上海交通大学）；University of Science and Technology Beijing（北京科技大学）

Tianyi Gao, Hao Li, Han Fang, Xin Wei, Xiaodong Dong, Hongbo Sun, Ye Yuan, Zhongjiang He, Jinglin Xu, Jingmin Xin, Hao Sun

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

RefBench-PRO提出一个面向感知与推理的指称表达理解基准，通过分解指称表达为感知和推理两个维度，设计六个逐步递增的挑战任务，并引入Ref-R1强化学习方案提升定位精度，实现对多模态大语言模型的可解释评估。

摘要：

指称表达理解（REC）是一种视觉-语言任务，其目标是根据文本描述定位特定图像区域。现有REC基准主要评估感知能力，缺乏可解释的评分机制，无法揭示多模态大语言模型（MLLM）在不同认知能力上的基础能力。为解决这一限制，我们引入RefBench-PRO，一个全面的REC基准，将指称表达分解为两个核心维度，即感知和推理，并进一步将其细分为六个逐步具有挑战性的任务，如属性、位置、交互、常识、关系和拒绝。我们还开发了一个完全自动化的数据生成管道，生成跨这些六个子维度的多样化指称表达。此外，我们提出了Ref-R1，一种基于强化学习的学习方案，结合动态IoU基于的GRPO以在日益复杂的推理条件下提高定位精度，建立更稳固的REC基准。广泛的实验表明，我们的RefBench-PRO能够对MLLM在指称表达理解上进行可解释的评估，同时在感知和推理方面都提出了更大的挑战。

英文摘要：

Referring Expression Comprehension (REC) is a vision-language task that localizes a specific image region based on a textual description. Existing REC benchmarks primarily evaluate perceptual capabilities and lack interpretable scoring mechanisms, which cannot reveal the grounding capability of Multi-modal Large Language Model (MLLM) across different cognitive abilities. To address this limitation, we introduce RefBench-PRO, a comprehensive REC benchmark, which decomposes referring expressions into two core dimensions, i.e., perception and reasoning, and further subdivides them into six progressively challenging tasks, such as attribute, position, interaction, commonsense, relation and reject. We also develop a fully automated data-generation pipeline that produces diverse referring expressions across these six sub-dimensions. Furthermore, We propose Ref-R1, an RL-based learning scheme, which incorporates Dynamic IoU-based GRPO to improve localization accuracy under increasingly complex reasoning conditions, establishing a stronger baseline for REC. Extensive experiments demonstrate that our RefBench-PRO enables interpretable evaluation of MLLM on referring expression comprehension, presenting greater challenges in both perception and reasoning.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。