ColLab：用于指代表达理解与生成的协作式空间渐进数据引擎
ColLab: A Collaborative Spatial Progressive Data Engine for Referring Expression Comprehension and Generation
AI总结：
针对REC与REG数据依赖人工标注、难以扩展的问题，本文提出ColLab全自动数据引擎，通过CMMI融合MLLM与LLM语义理解，并以SPA增强重复实例的空间表达，提升标注效率与表达质量。
摘要：
指代表达理解（REC）和指代表达生成（REG）是多模态理解中的基础任务，支持通过自然语言进行精确的目标定位。然而，现有REC和REG数据集严重依赖人工标注，劳动密集且难以扩展。本文提出ColLab，这是一种协作式空间渐进数据引擎，可在没有人工监督的情况下实现全自动REC和REG数据生成。具体而言，我们的方法引入了协作多模态模型交互（CMMI）策略，利用多模态大语言模型（MLLMs）和大语言模型（LLMs）的语义理解能力来生成描述。此外，我们设计了一个称为空间渐进增强（SPA）的模块，以增强重复实例之间的空间表达能力。实验表明，ColLab显著加快了REC和REG的标注过程，同时提高了所生成表达的质量和可区分性。除核心方法贡献外，我们的框架还被部分采用于ICCV 2025 MARS2多模态推理挑战赛的数据生成流程中，以多样且具有挑战性、更能反映真实世界推理需求的样本来丰富数据集。
英文摘要：
Referring Expression Comprehension (REC) and Referring Expression Generation (REG) are fundamental tasks in multimodal understanding, supporting precise object localization through natural language. However, existing REC and REG datasets rely heavily on manual annotation, which is labor-intensive and difficult to scale. In this paper, we propose ColLab, a collaborative spatial progressive data engine that enables fully automated REC and REG data generation without human supervision. Specifically, our method introduces a Collaborative Multimodal Model Interaction (CMMI) strategy, which leverages the semantic understanding of multimodal large language models (MLLMs) and large language models (LLMs) to generate descriptions. Furthermore, we design a module termed Spatial Progressive Augmentation (SPA) to enhance spatial expressiveness among duplicate instances. Experiments demonstrate that ColLab significantly accelerates the annotation process of REC and REG while improving the quality and discriminability of the generated expressions. In addition to the core methodological contribution, our framework was partially adopted in the data generation pipeline of the ICCV 2025 MARS2 Challenge on Multimodal Reasoning, enriching the dataset with diverse and challenging samples that better reflect real-world reasoning demands.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
