arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.04444cs.CV
收藏

参数高效语义增强用于提升开放词汇物体检测

Parameter-Efficient Semantic Augmentation for Enhancing Open-Vocabulary Object Detection

State Key Laboratory of Advanced Rail Autonomous Operation, Beijing Jiaotong University(北京交通大学先进轨道交通自主运行全国重点实验室);Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University(北京交通大学交通数据挖掘与具身智能北京市重点实验室);School of Computer Science & Technology, Beijing Jiaotong University(北京交通大学计算机科学与技术学院);WeChat AI, China(微信人工智能实验室)

Weihao Cao, Runqi Wang, Xiaoyue Duan, Jinchao Zhang, Ang Yang, Liping Jing

AI总结:

本文提出HSA-DINO框架,通过多尺度提示库和语义感知路由提升开放词汇物体检测的域适应性和泛化能力。

摘要:

开放词汇物体检测(OVOD)使模型能够检测任何物体类别,包括未见类别。得益于大规模预训练,现有OVOD方法在通用场景(如OV-COCO)中表现优异,但在转移至存在显著领域偏移的下游任务时性能急剧下降。这种退化源于领域特定任务中类别标签的稀缺性和弱语义,以及现有模型无法捕捉粗粒度类别标签之外的辅助语义。为解决这些问题,我们提出了HSA-DINO,一种参数高效的语义增强框架,用于增强开放词汇物体检测。具体而言,我们提出一个多尺度提示库,利用图像特征金字塔捕捉层次语义并选择领域特定的局部语义提示,逐步丰富从粗粒度到细粒度的文本表示。此外,我们引入了语义感知路由,动态选择合适的语义增强策略进行推理,从而防止参数更新影响预训练OVOD模型的泛化能力。我们在OV-COCO、多个垂直领域数据集和修改后的基准设置上评估了HSA-DINO。结果表明,HSA-DINO在域适应性和开放词汇泛化之间实现了更优的权衡,优于先前最先进的方法。

英文摘要:

Open-vocabulary object detection (OVOD) enables models to detect any object category, including unseen ones. Benefiting from large-scale pre-training, existing OVOD methods achieve strong detection performance on general scenarios (e.g., OV-COCO) but suffer severe performance drops when transferred to downstream tasks with substantial domain shifts. This degradation stems from the scarcity and weak semantics of category labels in domain-specific task, as well as the inability of existing models to capture auxiliary semantics beyond coarse-grained category label. To address these issues, we propose HSA-DINO, a parameter-efficient semantic augmentation framework for enhancing open-vocabulary object detection. Specifically, we propose a multi-scale prompt bank that leverages image feature pyramids to capture hierarchical semantics and select domain-specific local semantic prompts, progressively enriching textual representations from coarse to fine-grained levels. Furthermore, we introduce a semantic-aware router that dynamically selects the appropriate semantic augmentation strategy during inference, thereby preventing parameter updates from degrading the generalization ability of the pre-trained OVOD model. We evaluate HSA-DINO on OV-COCO, several vertical domain datasets, and modified benchmark settings. The results show that HSA-DINO performs favorably against previous state-of-the-art methods, achieving a superior trade-off between domain adaptability and open-vocabulary generalization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑