SCSA:探索空间注意力与通道注意力之间的协同效应
SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention
AI总结:
本文提出空间与通道协同注意力模块(SCSA),通过结合多语义空间注意力与渐进式通道自注意力,有效引导通道重校准并缓解语义差异,在多个视觉任务上超越现有方法。
摘要:
通道注意力和空间注意力在为各种下游视觉任务提取特征依赖性和空间结构关系方面分别带来了显著提升。虽然两者的结合更有利于发挥各自优势,但通道与空间注意力之间的协同作用尚未被充分探索,缺乏充分利用多语义信息进行特征引导和缓解语义差异的协同潜力。我们的研究试图揭示空间与通道注意力在多语义层次上的协同关系,提出了一种新颖的空间与通道协同注意力模块(SCSA)。我们的SCSA由两部分组成:可共享多语义空间注意力(SMSA)和渐进式通道自注意力(PCSA)。SMSA整合多语义信息,并利用渐进式压缩策略将判别性空间先验注入PCSA的通道自注意力中,有效引导通道重校准。此外,基于PCSA中自注意力机制的稳健特征交互,进一步缓解了SMSA内不同子特征之间多语义信息的差异。我们在七个基准数据集上进行了广泛实验,包括在ImageNet-1K上的分类、MSCOCO 2017上的目标检测、ADE20K上的分割以及其他四个复杂场景检测数据集。结果表明,我们提出的SCSA不仅超越了当前最先进的注意力机制,还在各种任务场景中展现出更强的泛化能力。代码和模型可在以下地址获取:https://github.com/HZAI-ZJNU/SCSA。
英文摘要:
Channel and spatial attentions have respectively brought significant improvements in extracting feature dependencies and spatial structure relations for various downstream vision tasks. While their combination is more beneficial for leveraging their individual strengths, the synergy between channel and spatial attentions has not been fully explored, lacking in fully harness the synergistic potential of multi-semantic information for feature guidance and mitigation of semantic disparities. Our study attempts to reveal the synergistic relationship between spatial and channel attention at multiple semantic levels, proposing a novel Spatial and Channel Synergistic Attention module (SCSA). Our SCSA consists of two parts: the Shareable Multi-Semantic Spatial Attention (SMSA) and the Progressive Channel-wise Self-Attention (PCSA). SMSA integrates multi-semantic information and utilizes a progressive compression strategy to inject discriminative spatial priors into PCSA's channel self-attention, effectively guiding channel recalibration. Additionally, the robust feature interactions based on the self-attention mechanism in PCSA further mitigate the disparities in multi-semantic information among different sub-features within SMSA. We conduct extensive experiments on seven benchmark datasets, including classification on ImageNet-1K, object detection on MSCOCO 2017, segmentation on ADE20K, and four other complex scene detection datasets. Our results demonstrate that our proposed SCSA not only surpasses the current state-of-the-art attention but also exhibits enhanced generalization capabilities across various task scenarios. The code and models are available at: https://github.com/HZAI-ZJNU/SCSA.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。