DRAgent：用于指代表达分割的判别推理智能体
DRAgent: Discriminative Reasoning Agent for Referring Expression Segmentation
- School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）
- Khoury College of Computer Sciences, Northeastern University（东北大学Khoury计算机科学学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文针对指代表达分割中MLLM单次坐标预测导致的定位偏差问题，提出DRAgent判别推理框架，通过两阶段目标选择与LoRA微调提升性能，在多个基准数据集上表现具竞争力。
AI中文摘要：
指代表达分割（RES）旨在为语言表达式指定的对象生成像素级掩码。近期基于多模态大语言模型（MLLM）的方法通常依赖单次坐标预测进行视觉定位，将连续空间位置序列化为离散文本标记，可能导致定位偏差和对齐错误。为解决这些问题，本文提出DRAgent，一种用于RES的MLLM驱动的判别推理（DR）框架。DRAgent不要求MLLM生成定位坐标，而是先构建检测器生成的候选空间，再将MLLM用作视觉-语义目标判别器。具体而言，MLLM通过两阶段DR机制在潜在干扰项中执行可靠的目标选择：第一阶段筛选高召回率候选，第二阶段进行实例级验证。所选目标框随后作为空间提示输入基础分割模型，以生成最终像素级掩码。此外，本文构建了经自一致性过滤的推理链数据流水线，用于基于LoRA的微调，为提升MLLM的判别推理能力提供更可靠的监督。实验表明，DRAgent在RefCOCO、RefCOCO+和RefCOCOg数据集上取得了具有竞争力的性能。
英文摘要：
Referring Expression Segmentation (RES) aims to generate a pixel-level mask for the object specified by a language expression. Recent methods based on multimodal large language models (MLLMs) often rely on one-pass coordinate prediction for visual localization, which serializes continuous spatial locations as discrete text tokens and may lead to localization bias and alignment errors. To address these issues, we propose DRAgent, an MLLM-driven discriminative reasoning (DR) framework for RES. Instead of requiring the MLLM to generate localization coordinates, DRAgent first constructs a detector-generated candidate space and then uses the MLLM as a visual-semantic target discriminator. Specifically, the MLLM performs reliable target selection among potential distractors through a two-stage DR mechanism, which first screens high-recall candidates and then performs instance-wise verification. The selected target box is subsequently used as a spatial prompt for a foundation segmentation model to produce the final pixel-level mask. Furthermore, we construct a self-consistency-filtered reasoning-chain data pipeline for LoRA-based fine-tuning, providing more reliable supervision for enhancing the MLLM's discriminative reasoning capability. Experiments demonstrate that DRAgent achieves competitive performance on RefCOCO, RefCOCO+, and RefCOCOg.
