CoT Referring: 通过 grounded 推理改进指称表达任务
CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning
AI总结：
通过 grounded 推理改进指称表达任务，提出CoT Referring方法，提升多模态大语言模型在复杂指称场景中的性能。
摘要：
指称表达理解与分割是评估语言理解和图像理解整合能力的关键任务，是多模态大语言模型（MLLMs）能力的基准。为解决这些挑战，我们提出了一种新的策略，CoT Referring，通过结构化的、链式思维训练数据结构增强模型跨模态的推理能力。我们的方法系统地解析文本结构为一个连续的指称步骤，在每一步中识别关系并确保参考对齐的一致性，从而在复杂查询场景中提高准确性。我们重构训练数据以强制新的输出形式，为现有数据集提供新的注释，并从现有资源中编纂一个评估基准。该基准专门设计用于复杂指称案例。我们还将检测和分割能力整合到统一的MLLM框架中，使用新的自适应加权损失进行训练以优化性能。在我们精心编纂的基准和RefCOCO/+/g上的实验结果表明了我们方法的有效性，与基线模型相比，有显著的2.5%+提升。
英文摘要：
Referring Expression Comprehension and Segmentation are critical tasks for assessing the integration of language understanding and image comprehension, serving as benchmarks for Multimodal Large Language Models (MLLMs) capabilities. To address these challenges, we propose a new strategy, CoT Referring, which enhances model reasoning across modalities through a structured, chain-of-thought training data structure. Our approach systematically parses textual structures to a sequential referring step, where in each step it identifies relationships and ensures consistent reference alignment, thereby improving accuracy in complex query scenarios. We restructure the training data to enforce a new output form, providing new annotations for existing datasets and compiling an evaluation benchmark from existing resources. This benchmark is designed explicitly for complex referring cases. We also integrate detection and segmentation capabilities into a unified MLLM framework, training it with a novel adaptive weighted loss to optimize performance. Experimental results on our curated benchmark and RefCOCO/+/g demonstrate the effectiveness of our approach, with a notable increase of 2.5%+ over baseline models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
