SKDF：一个用于将开放词汇知识蒸馏到开放世界目标检测器的简单知识蒸馏框架
SKDF: A Simple Knowledge Distillation Framework for Distilling Open-Vocabulary Knowledge to Open-world Object Detector
AI总结：
本文提出SKDF框架，通过降权损失函数和级联解耦解码结构将视觉-语言模型的开放词汇知识蒸馏到开放世界目标检测器，以缓解灾难性遗忘，并构建两个新基准验证未知目标检测性能。
摘要：
在本文中，我们尝试通过将视觉-语言模型（VLM）的开放世界知识蒸馏到一个语言无关的检测器中，使VLM模型专门用于开放世界目标检测（OWOD）任务。令人惊讶的是，我们观察到，简单的知识蒸馏方法与OWOD中的自动伪标签机制相结合，即使只使用少量数据，也能在未知目标检测上取得更好的性能。不幸的是，针对未知目标的知识蒸馏会严重影响采用传统结构的检测器对已知目标的学习，导致灾难性遗忘。为了缓解这些问题，我们提出了用于从视觉-语言模态向单一视觉模态进行知识蒸馏的降权损失函数。同时，我们提出了级联解耦解码结构，将定位学习与识别学习解耦，以减少已知和未知目标的类别交互对定位学习过程的影响。消融实验表明，这两种方法都能有效缓解开放世界知识蒸馏对已知目标学习的影响。此外，为了缓解当前缺乏用于评估开放世界检测器在开放世界中检测未知目标能力的全面基准的问题，我们基于测试场景的复杂度提出了两个基准，分别命名为“StandardSet”和“IntensiveSet”。在OWOD、MS-COCO以及我们提出的基准上进行的综合实验证明了我们方法的有效性。代码和所提出的数据集可在https://github.com/xiaomabufei/SKDF获取。
英文摘要：
In this paper, we attempt to specialize the VLM model for OWOD tasks by distilling its open-world knowledge into a language-agnostic detector. Surprisingly, we observe that the combination of a simple \textbf{knowledge distillation} approach and the automatic pseudo-labeling mechanism in OWOD can achieve better performance for unknown object detection, even with a small amount of data. Unfortunately, knowledge distillation for unknown objects severely affects the learning of detectors with conventional structures for known objects, leading to catastrophic forgetting. To alleviate these problems, we propose the \textbf{down-weight loss function} for knowledge distillation from vision-language to single vision modality. Meanwhile, we propose the \textbf{cascade decouple decoding structure} that decouples the learning of localization and recognition to reduce the impact of category interactions of known and unknown objects on the localization learning process. Ablation experiments demonstrate that both of them are effective in mitigating the impact of open-world knowledge distillation on the learning of known objects. Additionally, to alleviate the current lack of comprehensive benchmarks for evaluating the ability of the open-world detector to detect unknown objects in the open world, we propose two benchmarks, which we name "\textbf{StandardSet}$\heartsuit$" and "\textbf{IntensiveSet}$\spadesuit$" respectively, based on the complexity of their testing scenarios. Comprehensive experiments performed on OWOD, MS-COCO, and our proposed benchmarks demonstrate the effectiveness of our methods. The code and proposed dataset are available at \url{https://github.com/xiaomabufei/SKDF}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
