余弦归一化注意力用于超分辨率图像分类
Cosine-Normalized Attention for Hyperspectral Image Classification
AI总结：
本文提出余弦归一化注意力机制，通过几何视角优化超分辨率图像分类中的相似度计算，实验表明其在有限监督下优于多种Transformer和Mamba模型。
摘要：
基于Transformer的方法通过建模长距离空间-光谱依赖性提升了超分辨率图像分类（HSIC）性能；然而，其注意力机制通常依赖点积相似性，混合了特征幅度和方向，可能对超分辨率数据不适用。本文从几何视角重新审视注意力评分，引入余弦归一化注意力公式，将相似度计算与超分辨率签名的角结构对齐。通过将查询和键嵌入投影到单位超球面并应用平方余弦相似性，所提方法强调角关系并减少对幅度变化的敏感性。该公式被集成到空间-光谱Transformer中，并在极有限监督下进行评估。实验表明，所提方法在三个基准数据集上均取得更高性能，优于多种最近的Transformer和Mamba模型，尽管使用轻量级主干网络。此外，对多种注意力评分函数的受控分析显示，基于余弦的评分提供了可靠的归纳偏置用于超分辨率表示学习。
英文摘要：
Transformer-based methods have improved hyperspectral image classification (HSIC) by modeling long-range spatial-spectral dependencies; however, their attention mechanisms typically rely on dot-product similarity, which mixes feature magnitude and orientation and may be suboptimal for hyperspectral data. This work revisits attention scoring from a geometric perspective and introduces a cosine-normalized attention formulation that aligns similarity computation with the angular structure of hyperspectral signatures. By projecting query and key embeddings onto a unit hypersphere and applying a squared cosine similarity, the proposed method emphasizes angular relationships while reducing sensitivity to magnitude variations. The formulation is integrated into a spatial-spectral Transformer and evaluated under extremely limited supervision. Experiments on three benchmark datasets demonstrate that the proposed approach consistently achieves higher performance, outperforming several recent Transformer- and Mamba-based models despite using a lightweight backbone. In addition, a controlled analysis of multiple attention score functions shows that cosine-based scoring provides a reliable inductive bias for hyperspectral representation learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
