面向零样本标注目标检测的高效特征蒸馏
Efficient Feature Distillation for Zero-shot Annotation Object Detection
AI总结：
提出零样本标注目标检测设定 ZAD 及高效特征蒸馏方法 EZAD，通过重归一化 CLIP、生成含新类名的蒸馏建议并利用语义回归，在更短训练时间内提升 COCO 与 LVIS 性能。
摘要：
我们提出了一种用于检测未见目标的新设定，称为零样本标注目标检测（Zero-shot Annotation object Detection，ZAD）。它扩展了零样本目标检测设定，允许新类别目标出现在训练图像中，并将检测器使用的附加信息限制为新类别名称。近来，为了检测未见目标，不同方法利用了大规模视觉-语言模型（例如 CLIP）。基于蒸馏的方法具有良好的整体性能，但受到两个因素导致的漫长训练流程的困扰。首先，现有工作生成的蒸馏区域偏向基础类别，这限制了新类别信息的蒸馏。其次，直接使用来自 CLIP 的原始特征进行蒸馏忽视了 CLIP 训练数据与检测数据集之间的域差距，这使得学习从图像区域到视觉-语言特征空间的映射变得困难。为解决这些问题，我们提出了用于零样本标注目标检测的高效特征蒸馏方法（Efficient feature distillation for Zero-shot Annotation object Detection，EZAD）。首先，EZAD 通过重新归一化 CLIP，使其特征空间适应目标检测域；其次，EZAD 使用 CLIP 生成带有潜在新类别名称的蒸馏建议，以避免蒸馏过度偏向基础类别。最后，EZAD 利用语义信息进行回归，以进一步提升模型性能。结果，EZAD 在 COCO 上以短得多的训练流程超越此前基于蒸馏的方法 4%，并在 LVIS 数据集上取得 3% 的提升。我们的代码可在 https://github.com/dragonlzm/EZAD 获取。
英文摘要：
We propose a new setting for detecting unseen objects called Zero-shot Annotation object Detection (ZAD). It expands the zero-shot object detection setting by allowing the novel objects to exist in the training images and restricts the additional information the detector uses to novel category names. Recently, to detect unseen objects, large-scale vision-language models (e.g., CLIP) are leveraged by different methods. The distillation-based methods have good overall performance but suffer from a long training schedule caused by two factors. First, existing work creates distillation regions biased to the base categories, which limits the distillation of novel category information. Second, directly using the raw feature from CLIP for distillation neglects the domain gap between the training data of CLIP and the detection datasets, which makes it difficult to learn the mapping from the image region to the vision-language feature space. To solve these problems, we propose Efficient feature distillation for Zero-shot Annotation object Detection (EZAD). Firstly, EZAD adapts the CLIP's feature space to the target detection domain by re-normalizing CLIP; Secondly, EZAD uses CLIP to generate distillation proposals with potential novel category names to avoid the distillation being overly biased toward the base categories. Finally, EZAD takes advantage of semantic meaning for regression to further improve the model performance. As a result, EZAD outperforms the previous distillation-based methods in COCO by 4% with a much shorter training schedule and achieves a 3% improvement on the LVIS dataset. Our code is available at https://github.com/dragonlzm/EZAD
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
