整体迁移预训练Transformer编码器-解码器用于视觉目标检测
Integrally Migrating Pre-trained Transformer Encoder-decoders for Visual Object Detection
AI总结：
本研究提出imTED，整体迁移预训练Transformer编码器-解码器至检测器，移除随机初始化FPN并引入多尺度特征调制器，在MS COCO上提升2.4 AP，少样本检测提升7.6 AP。
摘要：
现代目标检测器利用了在大规模数据集上预训练的主干网络的优势。然而，除主干网络外，其他组件如检测头和特征金字塔网络（FPN）仍从零开始训练，这阻碍了充分挖掘表示模型的潜力。在本研究中，我们提出将预训练Transformer编码器-解码器（imTED）整体迁移到检测器中，构建一条“完全预训练”的特征提取路径，从而最大化检测器的泛化能力。imTED与基线检测器的主要区别有两方面：（1）将预训练Transformer解码器迁移到检测头，同时从特征提取路径中移除随机初始化的FPN；（2）定义多尺度特征调制器（MFM）以增强尺度适应性。这些设计不仅显著减少了随机初始化的参数，而且有意图地将检测器训练与表示学习统一起来。在MS COCO目标检测数据集上的实验表明，imTED始终优于对应方法，平均精度（AP）约高出2.4。无需额外技巧，imTED将少样本目标检测的最先进水平提升了高达7.6 AP。代码可在https://github.com/LiewFeng/imTED获取。
英文摘要：
Modern object detectors have taken the advantages of backbone networks pre-trained on large scale datasets. Except for the backbone networks, however, other components such as the detector head and the feature pyramid network (FPN) remain trained from scratch, which hinders fully tapping the potential of representation models. In this study, we propose to integrally migrate pre-trained transformer encoder-decoders (imTED) to a detector, constructing a feature extraction path which is ``fully pre-trained" so that detectors' generalization capacity is maximized. The essential differences between imTED with the baseline detector are twofold: (1) migrating the pre-trained transformer decoder to the detector head while removing the randomly initialized FPN from the feature extraction path; and (2) defining a multi-scale feature modulator (MFM) to enhance scale adaptability. Such designs not only reduce randomly initialized parameters significantly but also unify detector training with representation learning intendedly. Experiments on the MS COCO object detection dataset show that imTED consistently outperforms its counterparts by $\sim$2.4 AP. Without bells and whistles, imTED improves the state-of-the-art of few-shot object detection by up to 7.6 AP. Code is available at https://github.com/LiewFeng/imTED.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
