RSVG-ZeroOV：探索一种用于遥感图像零样本开放词汇视觉定位的无训练框架
RSVG-ZeroOV: Exploring a Training-Free Framework for Zero-Shot Open-Vocabulary Visual Grounding in Remote Sensing Images
AI总结：
针对现有遥感视觉定位方法闭集词汇限制及依赖昂贵数据集与微调的问题，提出无训练框架RSVG-ZeroOV，通过VLM跨注意力图、DM细粒度建模及注意力进化模块，实现零样本开放词汇定位，性能优于现有弱监督和零样本方法。
摘要：
遥感视觉定位（RSVG）旨在基于自由形式的自然语言表达式定位遥感图像中的目标。现有方法通常受限于闭集词汇，限制了其在开放世界场景中的适用性。尽管近期尝试利用通用基础模型解决开放词汇RSVG问题，但它们过度依赖昂贵的高质量数据集和耗时的微调。为解决这些局限，我们提出RSVG-ZeroOV，一种无训练框架，旨在探索冻结通用基础模型在零样本开放词汇RSVG中的潜力。具体而言，RSVG-ZeroOV包含三个关键阶段：（i）概览：我们利用视觉语言模型（VLM）获取跨注意力图，捕捉文本查询与视觉区域之间的语义关联。（ii）聚焦：通过利用扩散模型（DM）的细粒度建模先验，我们填补VLM常忽略的目标结构和形状信息空白。（iii）进化：引入简单且有效的注意力进化模块以抑制无关激活，生成目标对象的纯净分割掩码。无需繁琐的任务特定训练，RSVG-ZeroOV提供了高效且可扩展的解决方案。大量实验表明，该框架持续优于现有弱监督和零样本方法。
英文摘要：
Remote sensing visual grounding (RSVG) aims to localize objects in remote sensing images based on free-form natural language expressions. Existing approaches are typically constrained to closed-set vocabularies, limiting their applicability in open-world scenarios. While recent attempts to leverage generic foundation models for open-vocabulary RSVG, they overly rely on expensive high-quality datasets and time-consuming fine-tuning. To address these limitations, we propose \textbf{RSVG-ZeroOV}, a training-free framework that aims to explore the potential of frozen generic foundation models for zero-shot open-vocabulary RSVG. Specifically, RSVG-ZeroOV comprises three key stages: (i) Overview: We utilize a vision-language model (VLM) to obtain cross-attention\footnote[1]{In this paper, although decoder-only VLMs use self-attention over all tokens, we refer to the image-text interaction part as cross-attention to distinguish it from pure visual self-attention.}maps that capture semantic correlations between text queries and visual regions. (ii) Focus: By leveraging the fine-grained modeling priors of a diffusion model (DM), we fill in gaps in structural and shape information of objects, which are often overlooked by VLM. (iii) Evolve: A simple yet effective attention evolution module is introduced to suppress irrelevant activations, yielding purified segmentation masks over the referred objects. Without cumbersome task-specific training, RSVG-ZeroOV offers an efficient and scalable solution. Extensive experiments demonstrate that the proposed framework consistently outperforms existing weakly-supervised and zero-shot methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
