T-Rex-Omni:在通用目标检测中集成负视觉提示
T-Rex-Omni: Integrating Negative Visual Prompt in Generic Object Detection
AI总结:
针对开集目标检测中仅正提示易受视觉相似干扰物影响的问题,提出T-Rex-Omni框架,通过统一视觉提示编码器、NNC模块和NNH损失集成负视觉提示,提升零样本检测性能并缩小与文本提示方法的差距,在长尾场景表现突出。
摘要:
多年来,目标检测方法已从闭集范式发展到开集范式。然而,当前的开集目标检测器仍受限于仅依赖基于文本描述或视觉示例等给定提示的正指标。这种仅使用正样本的范式在面对视觉相似但语义不同的干扰物时持续存在脆弱性。我们提出T-Rex-Omni,这是一种通过纳入负视觉提示来否定难负干扰物的新颖框架。具体而言,我们首先引入一个统一的视觉提示编码器,联合处理正负视觉提示。接下来,提出一个无需训练的否定负计算(NNC)模块,在概率计算阶段动态抑制负响应。为通过微调进一步提升性能,我们的否定负铰链(NNH)损失在正负嵌入之间强制区分边界。T-Rex-Omni支持在仅正样本和联合正负样本推理模式下灵活部署,可适配用户指定或自动生成的负示例。大量实验展示了卓越的零样本检测性能,显著缩小了视觉提示方法与文本提示方法之间的性能差距,同时在长尾场景中表现出特别优势(在LVIS-minival上达到51.2 AP_r)。这项工作将负提示确立为推进开集视觉识别系统的关键新维度。
英文摘要:
Object detection methods have evolved from closed-set to open-set paradigms over the years. Current open-set object detectors, however, remain constrained by their exclusive reliance on positive indicators based on given prompts like text descriptions or visual exemplars. This positive-only paradigm experiences consistent vulnerability to visually similar but semantically different distractors. We propose T-Rex-Omni, a novel framework that addresses this limitation by incorporating negative visual prompts to negate hard negative distractors. Specifically, we first introduce a unified visual prompt encoder that jointly processes positive and negative visual prompts. Next, a training-free Negating Negative Computing (NNC) module is proposed to dynamically suppress negative responses during the probability computing stage. To further boost performance through fine-tuning, our Negating Negative Hinge (NNH) loss enforces discriminative margins between positive and negative embeddings. T-Rex-Omni supports flexible deployment in both positive-only and joint positive-negative inference modes, accommodating either user-specified or automatically generated negative examples. Extensive experiments demonstrate remarkable zero-shot detection performance, significantly narrowing the performance gap between visual-prompted and text-prompted methods while showing particular strength in long-tailed scenarios (51.2 AP_r on LVIS-minival). This work establishes negative prompts as a crucial new dimension for advancing open-set visual recognition systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。