arXiv 2603.279932026-03-30cs.CV

逐步引导的跨模态推理用于指代图像分割

Progressive Prompt-Guided Cross-Modal Reasoning for Referring Image Segmentation

School of Computer Science and Artificial Intelligence, Wuhan University of Technology（武汉理工大学计算机科学与人工智能学院）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；University of Electronic Science and Technology of China（电子科技大学）；Wuhan Vocational College of Software and Engineering（武汉软件工程职业学院）

Jiachen Li, Hongyun Wang, Jinyu Xu, Wenbo Jiang, Yanchun Ma, Yongjian Liu, Qing Xie, Bolong Zheng

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出PPCR框架，通过语义理解-空间定位-实例分割流程，改进指代图像分割中语言描述与视觉表示的连接，提升分割精度。

摘要：

指代图像分割旨在根据自由形式的指代表达在图像中定位和分割目标对象。核心挑战在于有效连接语言描述与对象级视觉表示，尤其当指代表达包含详细属性和复杂对象关系时。现有方法或依赖跨模态对齐或使用语义分割提示，但缺乏明确的推理机制将语言描述接地到图像中的目标区域。为解决这些限制，我们提出了PPCR，一种逐步引导的跨模态推理框架用于指代图像分割。PPCR将推理过程结构化为语义理解-空间定位-实例分割流程。具体而言，PPCR首先利用多模态大语言模型（MLLMs）生成语义分割提示，以捕捉目标对象的关键语义线索。基于此语义上下文，进一步生成空间分割提示以推理对象位置和空间范围，实现从语义理解到空间定位的逐步过渡。语义和空间分割提示随后被联合整合到分割模块中，以指导准确的目标定位和分割。在标准指代图像分割基准上的大量实验表明，PPCR在性能上始终优于现有方法。代码将公开发布以促进可重复性。

英文摘要：

Referring image segmentation aims to localize and segment a target object in an image based on a free-form referring expression. The core challenge lies in effectively bridging linguistic descriptions with object-level visual representations, especially when referring expressions involve detailed attributes and complex inter-object relationships. Existing methods either rely on cross-modal alignment or employ Semantic Segmentation Prompts, but they often lack explicit reasoning mechanisms for grounding language descriptions to target regions in the image. To address these limitations, we propose PPCR, a Progressive Prompt-guided Cross-modal Reasoning framework for referring image segmentation. PPCR explicitly structures the reasoning process as a Semantic Understanding-Spatial Grounding-Instance Segmentation pipeline. Specifically, PPCR first employs multimodal large language models (MLLMs) to generate Semantic Segmentation Prompt that capture key semantic cues of the target object. Based on this semantic context, Spatial Segmentation Prompt are further generated to reason about object location and spatial extent, enabling a progressive transition from semantic understanding to spatial grounding. The Semantic and Spatial Segmentation prompts are then jointly integrated into the segmentation module to guide accurate target localization and segmentation. Extensive experiments on standard referring image segmentation benchmarks demonstrate that PPCR consistently outperforms existing methods. The code will be publicly released to facilitate reproducibility.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。