空间调制协同注意下DETR的快速收敛性
Fast Convergence of DETR with Spatially Modulated Co-Attention
摘要：
最近提出的检测Transformer（DETR）模型成功地将Transformer应用于对象检测，并实现了与Faster-RCNN等两阶段目标检测框架相当的性能。然而，DETR的缺点是收敛速度慢。从头开始训练DETR需要500个epoch才能达到高精度。为了加快其收敛速度，我们提出了一个简单而有效的方案，即空间调制的协同注意力（SMCA）机制，以改善DETR框架。SMCA的核心思想是在DETR中通过将共同注意响应约束为在初始估计的限界框位置附近较高来进行位置感知的共同注意。我们提出的SMCA提高了DETR的收敛速度，取代原来的共同注意力机制在解码器中，而保持其他操作在DETR不变。此外，通过将多头和尺度选择注意力设计集成到SMCA中，我们完全成熟的SMCA可以实现比具有扩展的基于卷积的主干的DETR更好的性能（108个epochs时为45.6 mAP，500个epochs时为43.3 mAP）。我们对COCO数据集进行了广泛的消融研究，以验证SMCA。代码发布于https://github.com/gaopengcuhk/SMCA-DETR。
英文摘要：
The recently proposed Detection Transformer (DETR) model successfully applies Transformer to objects detection and achieves comparable performance with two-stage object detection frameworks, such as Faster-RCNN. However, DETR suffers from its slow convergence. Training DETR from scratch needs 500 epochs to achieve a high accuracy. To accelerate its convergence, we propose a simple yet effective scheme for improving the DETR framework, namely Spatially Modulated Co-Attention (SMCA) mechanism. The core idea of SMCA is to conduct location-aware co-attention in DETR by constraining co-attention responses to be high near initially estimated bounding box locations. Our proposed SMCA increases DETR's convergence speed by replacing the original co-attention mechanism in the decoder while keeping other operations in DETR unchanged. Furthermore, by integrating multi-head and scale-selection attention designs into SMCA, our fully-fledged SMCA can achieve better performance compared to DETR with a dilated convolution-based backbone (45.6 mAP at 108 epochs vs. 43.3 mAP at 500 epochs). We perform extensive ablation studies on COCO dataset to validate SMCA. Code is released at https://github.com/gaopengcuhk/SMCA-DETR .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
