arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.09455cs.LGmath.AT
收藏

简单注意力网络

Simplicial Attention Networks

University of Cambridge(剑桥大学)

Christopher Wei Jin Goh, Cristian Bodnar, Pietro Liò

摘要:

图表示学习方法大多局限于节点交互的建模。最近,人们越来越关注如何利用高阶结构来进一步增强组合空间中图神经网络(GNN)的学习能力。单纯神经网络(SNN)通过在单纯复合体(图的高维概括)上执行消息传递来自然地对这些交互进行建模。尽管如此,大多数现有SNN执行的计算都严格依赖于复合体的组合结构。利用注意力机制在结构化领域的成功,我们提出了单纯注意力网络(SAT),一种新型的单纯网络,动态权衡相邻单纯形之间的相互作用,可以很容易地适应新的结构。此外,我们提出了一个签署的注意力机制,使SAT方向等变,(共)链复合物上运行的模型的理想属性。我们证明了SAT在两个图像和轨迹分类任务中优于现有的卷积SNN和GNN。

英文摘要:

Graph representation learning methods have mostly been limited to the modelling of node-wise interactions. Recently, there has been an increased interest in understanding how higher-order structures can be utilised to further enhance the learning abilities of graph neural networks (GNNs) in combinatorial spaces. Simplicial Neural Networks (SNNs) naturally model these interactions by performing message passing on simplicial complexes, higher-dimensional generalisations of graphs. Nonetheless, the computations performed by most existent SNNs are strictly tied to the combinatorial structure of the complex. Leveraging the success of attention mechanisms in structured domains, we propose Simplicial Attention Networks (SAT), a new type of simplicial network that dynamically weighs the interactions between neighbouring simplicies and can readily adapt to novel structures. Additionally, we propose a signed attention mechanism that makes SAT orientation equivariant, a desirable property for models operating on (co)chain complexes. We demonstrate that SAT outperforms existent convolutional SNNs and GNNs in two image and trajectory classification tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑