FineCops-Ref：面向细粒度组合式指代表达理解的新数据集与任务
FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension
AI总结：
该研究构建了具备可控难度等级、含负样本的细粒度组合式REC新数据集FineCops-Ref，评估发现现有专用模型与MLLMs的定位性能仍有较大差距，旨在推动视觉推理与跨模态交互研究。
摘要：
指代表达理解（REC）是一项关键的跨模态任务，可客观评估语言理解、图像理解以及语言到图像的定位能力，因此是多模态大语言模型（MLLMs）的理想测试场景。为实现这一目标，我们构建了一个全新的REC数据集，它具备两大核心特征：第一，数据集设计了可控的不同难度等级，需要在物体类别、属性和多跳关系层面开展多级细粒度推理；第二，数据集包含基于现有数据通过细粒度编辑和生成得到的负样本文本与图像，用于测试模型正确拒绝图像中不存在目标物体场景的能力——这一关键维度在现有数据集和方法中常被忽视。借助该高质量数据集，我们对当前最优的专用模型和MLLMs开展了全面评估，结果显示，要达到令人满意的定位性能仍存在显著差距。我们期望该数据集能启发新方法以提升视觉推理能力、开发更先进的跨模态交互策略，最终充分释放MLLMs的潜力。我们的代码和数据集可在https://github.com/liujunzhuo/FineCops-Ref获取。
英文摘要：
Referring Expression Comprehension (REC) is a crucial cross-modal task that objectively evaluates the capabilities of language understanding, image comprehension, and language-to-image grounding. Consequently, it serves as an ideal testing ground for Multi-modal Large Language Models (MLLMs). In pursuit of this goal, we have established a new REC dataset characterized by two key features: Firstly, it is designed with controllable varying levels of difficulty, necessitating multi-level fine-grained reasoning across object categories, attributes, and multi-hop relationships. Secondly, it includes negative text and images created through fine-grained editing and generation based on existing data, thereby testing the model's ability to correctly reject scenarios where the target object is not visible in the image--an essential aspect often overlooked in existing datasets and approaches. Utilizing this high-quality dataset, we conducted comprehensive evaluations of both state-of-the-art specialist models and MLLMs. Our findings indicate that there remains a significant gap in achieving satisfactory grounding performance. We anticipate that our dataset will inspire new approaches to enhance visual reasoning and develop more advanced cross-modal interaction strategies, ultimately unlocking the full potential of MLLMs. Our code and the datasets are available at https://github.com/liujunzhuo/FineCops-Ref.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
