ProtoRAG：基于原型的检索增强用于少样本细粒度遥感目标检测

中文摘要 AI 辅助

遥感图像中的少样本细粒度目标检测（FGOD）具有挑战性，因为有限的标注必须同时支持目标定位和视觉上相似的子类别之间的区分。尽管多模态大语言模型（MLLMs）提供了强大的粗略目标定位能力，但它们缺乏可靠的细粒度识别所需的明确视觉证据。为了解决这一局限性，我们提出了ProtoRAG，一种基于原型的检索增强框架，通过为MLLMs配备外部对象级视觉记忆，将粗略定位与细粒度识别解耦。为了从有限的支撑样本中构建可靠的视觉记忆，我们引入了判别性原型空间学习（DPSL），通过监督对比学习和原型一致性正则化来鼓励判别性和原型稳定的表示。我们进一步开发了一种不确定性引导的候选约束推理策略，该策略用检索到的候选特定视觉参考增强MLLMs，并且仅对模糊实例调用多模态推理。大量实验表明，ProtoRAG在九个少样本设置中持续超越代表性基线，在MAR20、HRSC2016和FAIR1M-2.0上分别比最强基线高出14.80、2.27和4.04 mAP$_{50}$。

英文摘要

Few-shot fine-grained object detection (FGOD) in remote sensing imagery is challenging because limited annotations must support both object localization and discrimination among visually similar subcategories. Although multimodal large language models (MLLMs) provide strong coarse object localization, they lack explicit visual evidence for reliable fine-grained recognition. To address this limitation, we propose ProtoRAG, a prototype-based retrieval-augmented framework that decouples coarse localization from fine-grained recognition by equipping MLLMs with an external object-level visual memory. To construct a reliable visual memory from limited support samples, we introduce Discriminative Prototype Space Learning (DPSL), which encourages discriminative and prototype-stable representations through supervised contrastive learning and prototype-consistency regularization. We further develop an uncertainty-guided candidate-constrained reasoning strategy that augments MLLMs with retrieved candidate-specific visual references and invokes multimodal reasoning only for ambiguous instances. Extensive experiments show that ProtoRAG consistently surpasses representative baselines in nine few-shot settings, outperforming the strongest baselines by 14.80, 2.27, and 4.04 mAP$_{50}$ on MAR20, HRSC2016, and FAIR1M-2.0, respectively.

发表机构

• Wuhan University（武汉大学）
• East China Normal University（华东师范大学）

机构由 AI 辅助整理，请以论文原文为准。