VL-SAM-v3：基于记忆的开放世界目标检测视觉先验
VL-SAM-v3: Memory-Guided Visual Priors for Open-World Object Detection
AI总结：
VL-SAM-v3通过引入检索驱动的外部视觉记忆，提升开放世界目标检测的性能，尤其在稀有类别和复杂场景中表现优异。
摘要：
开放世界目标检测旨在定位和识别超出固定封闭集标签空间的对象。它通常分为两类：开放词汇检测，假设测试时有预定义类别列表；以及开放端检测，需在推理过程中生成候选类别。现有方法主要依赖粗粒度文本语义和参数化知识，常无法提供足够的视觉证据以处理细粒度外观变化、稀有类别和杂乱场景。本文提出VL-SAM-v3，一个统一框架，通过检索驱动的外部视觉记忆增强开放世界检测。具体而言，一旦候选类别可用，VL-SAM-v3从非参数化记忆库中检索相关视觉原型，并将其转换为两种互补的视觉先验：即稀疏先验用于实例级空间锚定，密集先验用于类别感知的局部上下文。这些先验通过记忆引导的提示细化与原始检测提示结合，实现共享的检索与细化机制，支持开放词汇和开放端推理。在LVIS上的广泛零样本实验表明，VL-SAM-v3在开放词汇和开放端推理中均能显著提升检测性能，尤其在稀有类别上表现突出。此外，与更强的开放词汇检测器（即SAM3）的实验验证了所提检索与细化机制的通用性。
英文摘要：
Open-world object detection aims to localize and recognize objects beyond a fixed closed-set label space. It is commonly divided into two categories, i.e., open-vocabulary detection, which assumes a predefined category list at test time, and open-ended detection, which requires generating candidate categories during the inference. Existing methods rely primarily on coarse textual semantics and parametric knowledge, which often provide insufficient visual evidence for fine-grained appearance variation, rare categories, and cluttered scenes. In this paper, we propose VL-SAM-v3, a unified framework that augments open-world detection with retrieval-grounded external visual memory. Specifically, once candidate categories are available, VL-SAM-v3 retrieves relevant visual prototypes from a non-parametric memory bank and transforms them into two complementary visual priors, i.e., sparse priors for instance-level spatial anchoring and dense priors for class-aware local context. These priors are integrated with the original detection prompts via Memory-Guided Prompt Refinement, enabling a shared retrieval-and-refinement mechanism that supports open-vocabulary and open-ended inference. Extensive zero-shot experiments on LVIS show that VL-SAM-v3 consistently improves detection performance under both open-vocabulary and open-ended inference, with particularly strong gains on rare categories. Moreover, experiments with a stronger open-vocabulary detector (i.e., SAM3) validate the generality of the proposed retrieval-and-refinement mechanism.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
