航空图像中的跨视角开放词汇目标检测
Cross-View Open-Vocabulary Object Detection in Aerial Imagery
AI总结：
针对航空图像开放词汇目标检测的跨域迁移难题，提出结合图像间对比对齐与多实例词汇关联的结构化域对齐框架，在多个航空数据集零样本任务中显著优于封闭词汇微调模型。
摘要：
传统目标检测模型通常基于固定类别集训练，灵活性受限，纳入新类别的成本高昂。开放词汇目标检测通过让模型无需专门训练即可识别未见过的类别，解决了这一局限。利用在大量可用的地面视角图像-文本分类对上进行对比训练的预训练模型，为航空图像中的开放词汇目标检测提供了坚实基础。域偏移、视角变化和极端的尺度差异使得跨域直接知识迁移效果不佳，需要专门的适配策略。本文提出了一种新颖框架，通过结构化域对齐，将地面视角图像的开放词汇表征适配用于解决航空图像中的目标检测问题。该方法引入对比式图像到图像对齐，以增强航空视角与地面视角嵌入之间的相似性，并采用多实例词汇关联将航空图像与文本嵌入对齐。我们在xView、DOTAv2、VisDrone、DIOR和HRRSD数据集上开展了大量实验以验证所提方法。在零样本设置下，我们的开放词汇模型与针对特定数据集微调的封闭词汇模型性能相比，在DOTAv2上取得+6.32 mAP的提升，在VisDrone（图像集）上提升+4.16 mAP，在HRRSD上提升+3.46 mAP，为航空应用中更灵活、可扩展的目标检测系统铺平了道路。
英文摘要：
Traditional object detection models are typically trained on a fixed set of classes, limiting their flexibility and making it costly to incorporate new categories. Open-vocabulary object detection addresses this limitation by enabling models to identify unseen classes without explicit training. Leveraging pretrained models contrastively trained on abundantly available ground-view image-text classification pairs provides a strong foundation for open-vocabulary object detection in aerial imagery. Domain shifts, viewpoint variations, and extreme scale differences make direct knowledge transfer across domains ineffective, requiring specialized adaptation strategies. In this paper, we propose a novel framework for adapting open-vocabulary representations from ground-view images to solve object detection in aerial imagery through structured domain alignment. The method introduces contrastive image-to-image alignment to enhance the similarity between aerial and ground-view embeddings and employs multi-instance vocabulary associations to align aerial images with text embeddings. Extensive experiments on the xView, DOTAv2, VisDrone, DIOR, and HRRSD datasets are used to validate our approach. Our open-vocabulary model achieves improvements of +6.32 mAP on DOTAv2, +4.16 mAP on VisDrone (Images), and +3.46 mAP on HRRSD in the zero-shot setting when compared to finetuned closed-vocabulary dataset-specific model performance, thus paving the way for more flexible and scalable object detection systems in aerial applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
