arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2201.11092cs.CV
收藏

自我注意神经特征袋

Self-Attention Neural Bag-of-Features

Tampere University(坦佩雷大学);Aarhus University(奥胡斯大学)

Kateryna Chumachenko, Alexandros Iosifidis, Moncef Gabbouj

摘要:

在这项工作中,我们提出了多变量序列数据的注意力公式。我们建立在最近推出的2D注意力的基础上,通过基于自我注意力的潜在空间量化特征/时间维度的相关性,而不是直接学习它们,从而重新制定注意力学习方法。此外,我们提出了一种联合特征-时间注意力机制,该机制学习联合2D注意力掩模,突出显示相关信息,而无需独立处理特征和时间表示。所提出的方法可以用于各种架构,我们专门评估其应用与神经袋的特征特征提取模块。几个序列数据分析任务的实验表明,我们的方法相比,标准方法产生的性能提高。

英文摘要:

In this work, we propose several attention formulations for multivariate sequence data. We build on top of the recently introduced 2D-Attention and reformulate the attention learning methodology by quantifying the relevance of feature/temporal dimensions through latent spaces based on self-attention rather than learning them directly. In addition, we propose a joint feature-temporal attention mechanism that learns a joint 2D attention mask highlighting relevant information without treating feature and temporal representations independently. The proposed approaches can be used in various architectures and we specifically evaluate their application together with Neural Bag of Features feature extraction module. Experiments on several sequence data analysis tasks show the improved performance yielded by our approach compared to standard methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑