AI 中文总结
该研究针对消息传递神经网络的虚拟节点内存瓶颈，提出可寻址且保留基数的虚拟内存方案，通过交叉注意力槽和私有锚点实现，在多重性感知等任务上验证了其有效性。
AI 中文摘要
虚拟节点为消息传递神经网络提供了一种简单的全局通信路径，但标准的“节点—虚拟节点—节点”流水线会将图压缩为一个同质状态，并将其相同地广播给每个节点。基于Mishayev等人的双半径分析，我们探究辅助虚拟内存如何在不使用自注意力的情况下缓解这种有限容量瓶颈。我们确定了两项要求：其一，全局内存应分解为可独立读写的状态，这可通过可寻址交叉注意力槽实现；其二，仅可寻址性无法保留多重性，因为softmax注意力对均匀复制具有不变性，插入每个槽查询作为私有键/值锚点可恢复被丢弃的归一化质量，且在有界颜色域上生成能实现1-WL细化的单射多重集表示。在多重性感知双半径、 motif计数及约束链接集预测任务上的实验表明，这种可寻址且保留基数的虚拟内存的算术成本为O(nMd)。
英文摘要
Virtual nodes are a natural tool against oversquashing: they replace long message-passing paths by a two-hop global route. But when many nodes share one global state, that shortcut can become a bottleneck itself. We study two properties of this global memory. First, addressability: under constant-margin address codes and a nonlinearity that amplifies this margin, multiplicative write/read maps provide $M$ selectable memory rows with only $O(\log M)$ address-code dimensions. Cross-attention slots and a constrained $ELU+1$ bilinear memory both satisfy these conditions. Second, support awareness: normalized cross-attention has no self-key for a latent query to use as a reference. A learned private anchor supplies this reference, keeps the read bounded, and exposes the strength of the matching source mass. We demonstrate the merits of such properties on several instances of Two-Radius and Tree-NeighborsMatch: both addressable realizations solve the controlled tasks through depth $5$, where pooled VNs of comparable or larger size reach about $10.6\%$.
Commentspreliminary work
