Conditional spatial queries are recently introduced into DEtection TRansformer (DETR) to accelerate convergence. In DAB-DETR, such queries are modulated by the so-called conditional linear projection at each decoder stage, aiming to search for positions of interest such as the four extremities of the box. Each decoder stage progressively updates the box by predicting the anchor box offsets, while in cross-attention only the box center is informed as the reference point. The use of only box center, however, leaves the width and height of the previous box unknown to the current stage, which hinders accurate prediction of offsets. We argue that the explicit use of the entire box information in cross-attention matters. In this work, we propose Box Agent to condense the box into head-specific agent points. By replacing the box center with the agent point as the reference point in each head, the conditional cross-attention can search for positions from a more reasonable starting point by considering the full scope of the previous box, rather than always from the previous box center. This significantly reduces the burden of the conditional linear projection. Experimental results show that the box agent leads to not only faster convergence but also improved detection performance, e.g., our single-scale model achieves $44.2$ AP with ResNet-50 based on DAB-DETR. Our Box Agent requires minor modifications to the code and has negligible computational workload. Code is available at https://github.com/tiny-smart/box-detr.
DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR
Tsinghua University(清华大学);International Digital Economy Academy (IDEA)(国际数字经济研究院(IDEA));Hong Kong University of Science and Technology(香港科技大学);Peng Cheng Laboratory(鹏城实验室)
SAP-DETR: Bridging the Gap Between Salient Points and Queries-Based Transformer Detector for Fast Model Convergency
Institute of Computing Technology (ICT), Chinese Academy of Sciences(中国科学院计算技术研究所);Stanford University(斯坦福大学);University of Chinese Academy of Sciences(中国科学院大学);AI Lab, Lenovo Research(联想研究院人工智能实验室);Lenovo Ltd.(联想集团有限公司)
PaQ-DETR: Learning Pattern and Quality-Aware Dynamic Queries for Object Detection
New York University(纽约大学);Boise State University(博伊西州立大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Irvine(加州大学 Irvine 分校);Illinois Institute of Technology(伊利诺伊理工学院);University of Pittsburgh(匹兹堡大学)
Dual-R-DETR: Resolving Query Competition with Pairwise Routing in Transformer Decoders
University of Pittsburgh(匹兹堡大学);University of California, Irvine(加州大学尔湾分校);Stevens Institute of Technology(史蒂文斯理工学院);Illinois Institute of Technology(伊利诺伊理工学院);New York University(纽约大学)
Robust Table Structure Recognition with Dynamic Queries Enhanced Detection Transformer
University of Science and Technology of China(中国科学技术大学);Microsoft Research Asia(微软亚洲研究院);Shanghai Jiao Tong University(上海交通大学);University of Chinese Academy of Sciences(中国科学院大学)
Decoupled DETR: Spatially Disentangling Localization and Classification for Improved End-to-End Object Detection
Multimedia Laboratory, The Chinese University of HongKong(香港中文大学多媒体实验室);SenseTime Research(商汤科技研究院);Centre for Perceptual and Interactive Intelligence Limited(感知与交互智能中心有限公司);Shanghai AI Laboratory(上海人工智能实验室)
College of Information Science and Engineering, Northeastern University(信息科学与工程学院,东北大学);State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University(过程工业综合自动化国家重点实验室,东北大学)
End-to-End Video Object Detection with Spatial-Temporal Transformers
Shanghai Jiao Tong University(上海交通大学);Qingyuan Research Institute, SJTU(上海交通大学清源研究院);Peking University(北京大学);Sensetime Research(商汤科技研究院);UCLA(加州大学洛杉矶分校)
Gaze-DETR: Top-Down Guidance Through Priority Maps for Infrared Weak-Small UAV Detection with DETR
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);ShanghaiTech University(上海科技大学);Beijing Institute of Basic Medical Sciences(北京基础医学研究所)
Frozen-DETR: Enhancing DETR with Image Understanding from Frozen Foundation Models
School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);Tongyi Lab, Alibaba Group(阿里巴巴集团通义实验室);Peng Cheng Laboratory(鹏城实验室);Pazhou Laboratory (Huangpu)(琶洲实验室(黄埔))
LP-DETR: Layer-wise Progressive Relations for Object Detection
New York University(纽约大学);University of Pittsburgh(匹兹堡大学);Fordham University(福特汉姆大学);Georgia Institute of Technology(佐治亚理工学院);California Institute of Technology(加州理工学院)
TAPTR: Tracking Any Point with Transformers as Detection
South China University of Technology(华南理工大学);International Digital Economy Academy (IDEA)(国际数字经济学院);The Hong Kong University of Science and Technology(香港科技大学);Tsinghua University(清华大学)
DExTeR: Weakly Semi-Supervised Object Detection with Class and Instance Experts for Medical Imaging
University of Strasbourg, CNRS, INSERM, ICube, UMR7357, France(斯特拉斯堡大学、法国国家科学研究中心、法国国家医学研究院、ICube、UMR7357、法国);University Hospital of Strasbourg, France(斯特拉斯堡大学医院、法国)