面向遥感图像与视频的无训练开放词汇视觉定位
Training-Free Open-Vocabulary Visual Grounding for Remote Sensing Images and Videos
AI总结：
提出无训练框架RSVG-ZeroOV，利用冻结的通用基础模型通过概览-聚焦-演化范式实现零样本开放词汇遥感视觉定位，并扩展至视频时空定位，在多个基准上超越现有零样本方法。
摘要：
遥感视觉定位（RSVG）旨在根据自然语言表达在遥感图像或视频中定位所指目标。现有的RSVG方法通常依赖于任务特定的手动标注，这些标注收集成本高昂，且在覆盖真实世界地理空间场景的多样性方面不可避免地存在局限。因此，它们往往难以泛化到涉及新物体、细粒度属性、复杂空间关系和功能语义的开放词汇查询。本文提出RSVG-ZeroOV，一个无训练框架，利用冻结的通用基础模型进行零样本开放词汇RSVG。RSVG-ZeroOV遵循概览-聚焦-演化范式，利用视觉语言模型（VLM）和扩散模型（DM）独特且互补的注意力模式逐步生成精确的定位结果。具体而言：(i) 概览利用VLM提取交叉注意力图，捕获指代表达与视觉区域之间的语义相关性；(ii) 聚焦利用DM的细粒度建模先验，补偿VLM注意力常忽略的物体结构和形状信息；(iii) 演化引入一个简单而有效的注意力演化模块，抑制无关激活，产生纯净的物体掩码。为处理视频输入，我们进一步提出Video RSVG-ZeroOV，通过查询相关关键帧选择器和时序传播器将图像级定位扩展到时空定位，无需视频标注或微调即可实现高效且时序一致的视频定位。在六个图像和视频定位基准上的大量实验表明，RSVG-ZeroOV持续优于现有零样本基线，并与弱监督和全监督方法相比达到有竞争力或更优的性能。
英文摘要：
Remote sensing visual grounding (RSVG) aims to localize a referred target in a remote sensing image or video according to a natural language expression. Existing RSVG methods usually rely on task-specific manual annotations, which are costly to collect and inevitably limited in covering the diversity of real-world geospatial scenarios. As a result, they often struggle to generalize to open-vocabulary queries involving novel objects, fine-grained attributes, complex spatial relationships, and functional semantics. In this paper, we propose RSVG-ZeroOV, a training-free framework that leverages frozen generic foundation models for zero-shot open-vocabulary RSVG. RSVG-ZeroOV follows an Overview-Focus-Evolve paradigm, which exploits the distinct yet complementary attention patterns of vision-language models (VLMs) and diffusion models (DMs) to progressively generate precise grounding results. Specifically, (i) Overview utilizes a VLM to extract cross-attention maps that capture semantic correlations between the referring expression and visual regions; (ii) Focus leverages the fine-grained modeling priors of a DM to compensate for object structure and shape information often overlooked by VLM attention; and (iii) Evolve introduces a simple yet effective attention evolution module to suppress irrelevant activations, yielding purified object masks. To handle video inputs, we further present Video RSVG-ZeroOV, which extends image-level grounding to spatio-temporal grounding through a query-relevant key-frame selector and a temporal propagator, enabling efficient and temporally coherent video grounding without video annotations or fine-tuning. Extensive experiments on six image and video grounding benchmarks show that RSVG-ZeroOV consistently outperforms existing zero-shot baselines and achieves competitive or superior performance compared with weakly- and fully-supervised methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
