ProVG:通过语言解耦实现遥感图像的渐进式视觉定位
ProVG: Progressive Visual Grounding via Language Decoupling for Remote Sensing Imagery
AI总结:
ProVG通过解耦语言表达为全局上下文、空间关系和对象属性,提出一种改进遥感图像定位精度的新框架,采用渐进式跨模态调节器和跨尺度融合模块,实现更精确的视觉语言对齐。
摘要:
遥感视觉定位(RSVG)旨在根据自然语言表达在遥感图像中定位物体。先前方法通常依赖于句子级的视觉-语言对齐,难以利用细粒度语言线索,如空间关系和对象属性,这些线索对区分特征相似的物体至关重要。重要的是,这些线索在不同的定位阶段扮演着不同的角色,应相应利用以提供更明确的指导。在本文中,我们提出ProVG,一种新的RSVG框架,通过将语言表达解耦为全局上下文、空间关系和对象属性来提高定位精度。为了整合这些语言线索,ProVG采用了一种简单而有效的渐进式跨模态调节器,通过调查-定位-验证方案动态调节视觉注意力,实现粗到细的视觉-语言对齐。此外,ProVG还集成了跨尺度融合模块以缓解遥感图像中的大尺度变化,并结合语言引导的校准解码器在预测过程中优化跨模态对齐。统一的多任务头进一步使ProVG能够支持指称表达理解和分割任务。在两个基准测试集上进行的广泛实验表明,ProVG在多个方面均优于现有方法,实现了新的最先进的性能。
英文摘要:
Remote sensing visual grounding (RSVG) aims to localize objects in remote sensing imagery according to natural language expressions. Previous methods typically rely on sentence-level vision-language alignment, which struggles to exploit fine-grained linguistic cues, such as \textit{spatial relations} and \textit{object attributes}, that are crucial for distinguishing objects with similar characteristics. Importantly, these cues play distinct roles across different grounding stages and should be leveraged accordingly to provide more explicit guidance. In this work, we propose \textbf{ProVG}, a novel RSVG framework that improves localization accuracy by decoupling language expressions into global context, spatial relations, and object attributes. To integrate these linguistic cues, ProVG employs a simple yet effective progressive cross-modal modulator, which dynamically modulates visual attention through a \textit{survey-locate-verify} scheme, enabling coarse-to-fine vision-language alignment. In addition, ProVG incorporates a cross-scale fusion module to mitigate the large-scale variations in remote sensing imagery, along with a language-guided calibration decoder to refine cross-modal alignment during prediction. A unified multi-task head further enables ProVG to support both referring expression comprehension and segmentation tasks. Extensive experiments on two benchmarks, \textit{i.e.}, RRSIS-D and RISBench, demonstrate that ProVG consistently outperforms existing methods, achieving new state-of-the-art performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。