通过微调 CLIP 实现开放词汇 X 射线违禁物品检测
Open-Vocabulary X-ray Prohibited Item Detection via Fine-tuning CLIP
AI总结：
针对 X 射线安检中新增违禁品类别识别难题，本文提出带瓶颈适配器的 OVXD 模型，通过微调 CLIP 对齐 X 射线图像与文本，并在 PIXray、PIDray 上显著提升新类别检测性能。
摘要：
X 射线违禁物品检测是安全检查的重要组成部分，并且违禁物品类别会根据最新法律法规持续增加。以往工作均聚焦于闭集场景，只能识别用于训练的已知类别，在学习新类别时往往需要耗时且劳动密集的标注，导致现实应用受限。尽管视觉语言模型（例如 CLIP）的成功为开集 X 射线违禁物品检测提供了新视角，但由于 X 射线数据与用于预训练 CLIP 的通用数据之间存在域偏移，将 CLIP 直接应用于 X 射线领域会造成性能急剧下降。为应对上述挑战，本文将基于蒸馏的开放词汇目标检测（OVOD）任务引入 X 射线安全检查领域，通过扩展 CLIP 来学习特定 X 射线领域中的视觉表示，旨在检测检测器所训练的基础类别之外的新型违禁物品类别。具体而言，我们提出 X 射线特征适配器，并在 OVOD 框架内将其应用于 CLIP，以开发 OVXD 模型。X 射线特征适配器包含三个采用瓶颈架构的适配器子模块，结构简单，却能高效整合 X 射线领域新知识与原始知识，进一步弥合领域差距，并促进 X 射线图像与文本概念之间的对齐。在 PIXray 和 PIDray 数据集上进行的大量实验表明，所提方法在 X 射线场景下检测新类别时表现优于其他 OVOD 基线方法。该方法在 PIXray 和 PIDray 上分别取得 21.0 AP50 和 27.8 AP50，较此前最佳结果分别提升 15.2 AP50 和 1.5 AP50。
英文摘要：
X-ray prohibited item detection is an essential component of security check and categories of prohibited item are continuously increasing in accordance with the latest laws. Previous works all focus on close-set scenarios, which can only recognize known categories used for training and often require time-consuming as well as labor-intensive annotations when learning novel categories, resulting in limited real-world applications. Although the success of vision-language models (e.g. CLIP) provides a new perspectives for open-set X-ray prohibited item detection, directly applying CLIP to X-ray domain leads to a sharp performance drop due to domain shift between X-ray data and general data used for pre-training CLIP. To address aforementioned challenges, in this paper, we introduce distillation-based open-vocabulary object detection (OVOD) task into X-ray security inspection domain by extending CLIP to learn visual representations in our specific X-ray domain, aiming to detect novel prohibited item categories beyond base categories on which the detector is trained. Specifically, we propose X-ray feature adapter and apply it to CLIP within OVOD framework to develop OVXD model. X-ray feature adapter containing three adapter submodules of bottleneck architecture, which is simple but can efficiently integrate new knowledge of X-ray domain with original knowledge, further bridge domain gap and promote alignment between X-ray images and textual concepts. Extensive experiments conducted on PIXray and PIDray datasets demonstrate that proposed method performs favorably against other baseline OVOD methods in detecting novel categories in X-ray scenario. It outperforms previous best result by 15.2 AP50 and 1.5 AP50 on PIXray and PIDray with achieving 21.0 AP50 and 27.8 AP50 respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
