DETR has set up a simple end-to-end pipeline for object detection by formulating this task as a set prediction problem, showing promising potential. Despite its notable advancements, this paper identifies two key forms of misalignment within the model: classification-regression misalignment and cross-layer target misalignment. Both issues impede DETR's convergence and degrade its overall performance. To tackle both issues simultaneously, we introduce a novel loss function, termed as Align Loss, designed to resolve the discrepancy between the two tasks. Align Loss guides the optimization of DETR through a joint quality metric, strengthening the connection between classification and regression. Furthermore, it incorporates an exponential down-weighting term to facilitate a smooth transition from positive to negative samples. Align-DETR also employs many-to-one matching for supervision of intermediate layers, akin to the design of H-DETR, which enhances robustness against instability. We conducted extensive experiments, yielding highly competitive results. Notably, our method achieves a 49.3% (+0.6) AP on the H-DETR baseline with the ResNet-50 backbone. It also sets a new state-of-the-art performance, reaching 50.5% AP in the 1x setting and 51.7% AP in the 2x setting, surpassing several strong competitors. Our code is available at https://github.com/FelixCaae/AlignDETR.
Semantic-Aligned Matching for Enhanced DETR Convergence and Multi-Scale Feature Fusion
Nanyang Technological University(南洋理工大学);Carnegie Mellon University(卡内基梅隆大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
最近提出的检测Transformer(DETR)建立了一个完全端到端的范例目标检测。然而,DETR遭受缓慢的训练收敛,这阻碍了其适用于各种检测任务。我们观察到,DETR的缓慢收敛在很大程度上是由于对象查询和编码的图像特征之间的语义不一致,在匹配对象查询到相关区域的困难。基于此,我们设计了语义对齐匹配DETR++(SAM-DETR++)来加速DETR的收敛,提高检测性能。SAM-DETR++的核心是一个即插即用的模块,它将对象查询和编码的图像特征投影到同一个特征嵌入空间中,每个对象查询都可以轻松地匹配到具有相似语义的相关区域。此外,SAM-DETR++搜索多个代表性的关键点,并利用其特征进行语义对齐匹配,增强了表示能力。此外,SAM-DETR++可以有效地融合多尺度特征,在设计的语义对齐匹配的基础上,以粗到细的方式。大量的实验表明,所提出的SAM-DETR++实现了优越的收敛速度和竞争力的检测精度。此外,作为一种即插即用的方法,SAM-DETR++可以补充现有的DETR融合解决方案,具有更好的性能,在COCO val 2017上使用ResNet-50,仅用12个训练epoch就实现了44.8%的AP,用50个训练epoch实现了49.1%的AP。代码可在https://github.com/ZhangGongjie/SAM-DETR上获得。
DeCo-DETR: Decoupled Cognition DETR for efficient Open-Vocabulary Object Detection
Jiangsu University(江苏大学);Brown University(布朗大学);Nanyang Technological University(南洋理工大学);MBZUAI;University of New South Wales(新南威尔士大学);USC;University of Toronto(多伦多大学);Data61 CSIRO
Concealed Object Detection for Passive Millimeter-Wave Security Imaging Based on Task-Aligned Detection Transformer
Huazhong University of Science and Technology(华中科技大学);National Key Laboratory of Science and Technology on Multi-Spectral Information Processing(多光谱信息处理科学与技术国家重点实验室)
School of Electronic Information and Communications, Huazhong University of Science and Technology(华中科技大学电子信息与通信学院);AAII, University of Technology Sydney(悉尼科技大学人工智能高级研究院);UIUC(伊利诺伊大学厄巴纳-香槟分校);University of Oregon(俄勒冈大学);Picsart AI Research(Picsart人工智能研究院)
Gaze-DETR: Using Expert Gaze to Reduce False Positives in Vulvovaginal Candidiasis Screening
State Key Laboratory of Advanced Medical Materials and Devices(先进医用材料与器件国家重点实验室);School of Biomedical Engineering, Shanghai Jiao Tong University(上海交通大学生物医学工程学院);Shanghai Clinical Research and Trial Center(上海临床研究与试验中心)