发表机构
University of Electronic Science and Technology of China（电子科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对脉冲Transformer因低通滤波导致高频信息丢失的性能瓶颈，本文提出受生物视觉启发的脉冲对比注意力（SCA）范式，通过全局对比聚合与局部差分细化增强高频分量，在多种任务上显著提升性能并降低复杂度。
AI 中文摘要
脉冲Transformer将脉冲神经网络（SNNs）的能效优势与自注意力的表征能力相结合，为高性能、高能效计算构建了一种前景广阔的网络架构。然而，其性能与人工神经网络（ANNs）中的对应架构相比仍存在差距。与先前将其归因于二值激活的研究不同，我们通过多尺度频谱分析揭示，脉冲神经元和脉冲自注意力（SSA）均充当低通滤波器。这一特性导致高频分量的耗散。为解决此问题，我们借鉴生物视觉系统的边缘检测和差分感知特性，提出了脉冲对比注意力（SCA）范式。通过全局对比聚合提取对比原型，并应用局部差分细化，SCA有效增强了高频信息。大量实验表明，SCA是一种通用模块，能在图像分类、语义分割和基于事件的跟踪任务中持续提升脉冲Transformer的性能。此外，它实现了更低的复杂度，相较于原始SSA提供了更优的效率。这些结果确立了其作为高能效脉冲Transformer基础构建模块的潜力。
英文摘要
Spiking Transformers merge the energy-efficiency of spiking neural networks (SNNs) with the representational power of self-attention, creating a promising architecture for high-performance, energy-efficient computation. However, a performance gap persists versus its counterparts in artificial neural networks (ANNs). Unlike prior works attributing this to binary activations, we reveal that both spiking neurons and spiking self-attention (SSA) act as low-pass filters through multiscale spectral analysis. This characteristic leads to the dissipation of high-frequency components. To address this issue, we propose the Spiking Contrastive Attention (SCA) paradigm, which draw inspiration from the edge-detection and differential sensing properties of biological visual system. By extracting contrast prototypes via global contrastive aggregation and applying local differential refinement, SCA effectively enhances high-frequency information. Extensive experiments show that SCA is a general module that consistently boosts Spiking Transformers across image classification, semantic segmentation, and event-based tracking. Furthermore, it achieves lower complexity, offering superior efficiency over original SSA. These results establish its potential as a fundamental building block for energy-efficient Spiking Transformers.
CommentsSpiking Neural Networks
