DR$^2$Seg: 分解式两阶段 rollout 用于多模态大语言模型中的高效推理分割
DR$^2$Seg: Decomposed Two-Stage Rollouts for Efficient Reasoning Segmentation in Multimodal Large Language Models
AI总结：
DR$^2$Seg通过分解式两阶段rollout策略提升多模态大语言模型中的推理效率和分割准确性。
摘要：
推理分割是一种新兴的视觉-语言任务，需要对复杂的文本查询进行推理以精确分割对象。然而，现有方法通常存在过度思考的问题，生成冗长的推理链，干扰多模态大语言模型（MLLMs）中的对象定位。为了解决这个问题，我们提出了DR$^2$Seg，一种自奖励框架，能够在不需额外思考监督的情况下提升推理效率和分割准确性。DR$^2$Seg采用两阶段rollout策略，将推理分割分解为多模态推理和指引用分割。在第一阶段，模型生成一个自包含的描述，明确指定目标对象。在第二阶段，该描述替代原始复杂查询以验证其自包含性。基于此设计，引入了两种自奖励以缓解过度思考及相关的注意力分散。在Qwen2.5-VL的3B和7B变体以及SAM2和SAM3上进行的大量实验表明，DR$^2$Seg一致地提升了推理效率和整体分割准确性。
英文摘要：
Reasoning segmentation is an emerging vision-language task that requires reasoning over intricate text queries to precisely segment objects. However, existing methods typically suffer from overthinking, generating verbose reasoning chains that interfere with object localization in multimodal large language models (MLLMs). To address this issue, we propose DR$^2$Seg, a self-rewarding framework that improves both reasoning efficiency and segmentation accuracy without requiring extra thinking supervision. DR$^2$Seg employs a two-stage rollout strategy that decomposes reasoning segmentation into multimodal reasoning and referring segmentation. In the first stage, the model generates a self-contained description that explicitly specifies the target object. In the second stage, this description replaces the original complex query to verify its self-containment. Based on this design, two self-rewards are introduced to mitigate overthinking and the associated attention dispersion. Extensive experiments conducted on 3B and 7B variants of Qwen2.5-VL, as well as on both SAM2 and SAM3, demonstrate that DR$^2$Seg consistently improves reasoning efficiency and overall segmentation accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
