超越开放词汇：面向遥感图像的目标检测多模态提示
Beyond Open Vocabulary: Multimodal Prompting for Object Detection in Remote Sensing Images
AI总结：
RS-MPOD提出了一种多模态开放词汇检测框架，通过整合视觉和文本提示提升遥感图像中目标检测的稳定性与准确性。
摘要：
遥感图像中的开放词汇目标检测通常依赖于仅文本的提示来指定目标类别，隐含假设在推理时的类别查询可以通过预训练诱导的文本-视觉对齐可靠地 grounding。在实践中，由于任务和应用特定的类别语义，这一假设在遥感场景中经常失效，导致开放词汇设置下的类别指定不稳定。为了解决这一限制，我们提出了RS-MPOD，一种多模态开放词汇检测框架，通过整合实例接地的视觉提示、文本提示及其多模态整合，重新定义类别指定，超越仅文本的提示。RS-MPOD引入了视觉提示编码器，从示例实例中提取基于外观的类别线索，实现无文本的类别指定，并引入多模态融合模块，在两种模态都可用时整合视觉和文本信息。在标准、跨数据集和细粒度遥感基准上的广泛实验表明，视觉提示在语义模糊和分布偏移时提供更可靠的类别指定，而多模态提示在文本语义对齐良好时提供灵活的替代方案，保持竞争力。
英文摘要：
Open-vocabulary object detection in remote sensing commonly relies on text-only prompting to specify target categories, implicitly assuming that inference-time category queries can be reliably grounded through pretraining-induced text-visual alignment. In practice, this assumption often breaks down in remote sensing scenarios due to task- and application-specific category semantics, resulting in unstable category specification under open-vocabulary settings. To address this limitation, we propose RS-MPOD, a multimodal open-vocabulary detection framework that reformulates category specification beyond text-only prompting by incorporating instance-grounded visual prompts, textual prompts, and their multimodal integration. RS-MPOD introduces a visual prompt encoder to extract appearance-based category cues from exemplar instances, enabling text-free category specification, and a multimodal fusion module to integrate visual and textual information when both modalities are available. Extensive experiments on standard, cross-dataset, and fine-grained remote sensing benchmarks show that visual prompting yields more reliable category specification under semantic ambiguity and distribution shifts, while multimodal prompting provides a flexible alternative that remains competitive when textual semantics are well aligned.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
