arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1910.03151cs.CV
收藏

ECA-Net:深度卷积神经网络的高效通道注意

ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

Tianjin University(天津大学);Dalian University of Technology(大连理工大学);Harbin Institute of Technology(哈尔滨工业大学)

Qilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, Qinghua Hu

摘要:

最近,通道注意力机制已被证明在提高深度卷积神经网络(CNN)的性能方面具有巨大的潜力。然而,大多数现有方法都致力于开发更复杂的注意力模块以获得更好的性能,这不可避免地增加了模型的复杂度。为了克服性能和复杂性权衡的矛盾,本文提出了一个有效的通道注意力(ECA)模块,它只涉及少数参数,同时带来明显的性能增益。通过剖析SENet中的通道注意力模块,我们经验性地表明,避免降维对于学习通道注意力非常重要,并且适当的跨通道交互可以在显著降低模型复杂度的同时保持性能。因此,我们提出了一个本地跨信道的互动策略,没有降维,这可以有效地实现通过$1D$卷积。此外,我们开发了一种方法来自适应地选择$1D$卷积的核大小,确定局部跨通道交互的覆盖范围。拟议的非洲经委会模块既有效率又有效,例如,我们的模块对ResNet 50的主干的参数和计算分别为80 vs. 24.37M和4.7e-4 GFLOPs vs. 3.86 GFLOPs,并且在Top-1精度方面性能提升超过2%。我们广泛评估我们的ECA模块的图像分类,目标检测和实例分割与ResNets和MobileNetV 2的骨干。实验结果表明,我们的模块是更有效的,同时表现良好,对它的同行。

英文摘要:

Recently, channel attention mechanism has demonstrated to offer great potential in improving the performance of deep convolutional neural networks (CNNs). However, most existing methods dedicate to developing more sophisticated attention modules for achieving better performance, which inevitably increase model complexity. To overcome the paradox of performance and complexity trade-off, this paper proposes an Efficient Channel Attention (ECA) module, which only involves a handful of parameters while bringing clear performance gain. By dissecting the channel attention module in SENet, we empirically show avoiding dimensionality reduction is important for learning channel attention, and appropriate cross-channel interaction can preserve performance while significantly decreasing model complexity. Therefore, we propose a local cross-channel interaction strategy without dimensionality reduction, which can be efficiently implemented via $1D$ convolution. Furthermore, we develop a method to adaptively select kernel size of $1D$ convolution, determining coverage of local cross-channel interaction. The proposed ECA module is efficient yet effective, e.g., the parameters and computations of our modules against backbone of ResNet50 are 80 vs. 24.37M and 4.7e-4 GFLOPs vs. 3.86 GFLOPs, respectively, and the performance boost is more than 2% in terms of Top-1 accuracy. We extensively evaluate our ECA module on image classification, object detection and instance segmentation with backbones of ResNets and MobileNetV2. The experimental results show our module is more efficient while performing favorably against its counterparts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑