AI 中文总结
本研究通过箭图表示将层神经网络中的矩阵值传输与多头注意力联系起来，提出拓扑注意力（Top-A），学习跨头非对角路由，在关系推理等任务上验证了边条件跨头通信作为独立计算原语的价值。
AI 中文摘要
层神经网络通过用局部特征空间之间的线性传输映射替换标量边权重，推广了标量加权消息传递。然而，这种矩阵值传输的作用与更广泛的层扩散结构纠缠在一起。我们通过箭图表示分离出传输原语，并建立了与多头注意力的直接联系。将注意力头视为局部传输空间的坐标，揭示了标准多头注意力实现的是对角边映射：沿着每个有向交互，源头只能贡献给对应的接收头。允许非对角元素反而能够在邻域聚合之前实现跨头的边条件通信。我们证明，这种操作通常不能被吸收到聚合后应用的单一共享线性映射中。基于这一表征，我们引入了拓扑注意力（Top-A），一种多头注意力，它学习依赖于边的非对角路由，同时保留原始的相同头路径，并在额外路由消失时精确恢复普通注意力。我们在关系推理、异质图学习和算法推理（包括分布外泛化）上评估了Top-A，并以异质节点分类作为对比设置。结果表明，当任务受益于交互依赖的变换时，跨头传输最为有用，而仅异质性本身不提供系统性优势。这些发现将边条件的跨头通信确定为矩阵值传输的一个独特计算原语。
英文摘要
Sheaf Neural Networks generalize scalar-weighted message passing by replacing scalar edge weights with linear transport maps between local feature spaces. Yet the role of this matrix-valued transport is entangled with the broader sheaf-diffusion construction. We isolate the transport primitive through quiver representations and establish a direct connection with multi-head attention. Treating attention heads as coordinates of a local transport space reveals that standard multi-head attention implements diagonal edge maps: along each directed interaction, a source head can contribute only to the corresponding receiver head. Allowing off-diagonal entries instead enables edge-conditioned communication across heads before neighborhood aggregation. We show that this operation cannot, in general, be absorbed into a single shared linear map applied after aggregation. Building on this characterization, we introduce Topological Attention (Top-A), a multi-head attention that learns edge-dependent off-diagonal routes while preserving the original same-head paths and exactly recovering vanilla attention when the additional routing vanishes. We evaluate Top-A on relational reasoning, heterogeneous graph learning, and algorithmic reasoning, including out-of-distribution generalization, with heterophilic node classification as a contrast setting. The results show that cross-head transport is most useful when the task benefits from interaction-dependent transformations, while heterophily alone provides no systematic advantage. These findings identify edge-conditioned cross-head communication as a distinct computational primitive of matrix-valued transport.
