AI 中文总结
提出AW-RoPE，将旋转置于边上并沿所有游走求和，以区分图路径，在合成和真实基准上显著降低nRMSE并取得最佳性能。
AI 中文摘要
旋转位置编码使注意力对相对位置敏感，但将其扩展到图需要选择图结构如何进入旋转。先前的工作为每个节点分配来自谱坐标的旋转，因此两个节点之间的旋转因子仅取决于它们的端点，无法区分连接它们的路径。我们引入了分析游走旋转位置编码（AW-RoPE），将旋转置于边上，并对所有游走求和传输的特征，从而使不同路径上的贡献可以相互增强或抵消。精确变体通过可微线性求解评估完整和，稀疏变体在有限深度处截断。我们证明了在固定输入和参数下前向和参数导数的截断界。两种变体作用于投影的查询和键，稀疏递推还增强了消息传递网络。在五个合成任务中，两种变体相对于最强基线将nRMSE降低了15%–58%，在真实超像素、肽和OGB基准上，稀疏递推在Performer核的每个数据集上以及GIN的十三个数据集中的十二个上取得了最佳平均值。分析表明，AW-RoPE可以区分仅以两个端点如何连接为线索的路径，而节点式旋转编码则无法做到。
英文摘要
Rotary position encodings make attention sensitive to relative position, but extending them to graphs requires choosing how graph structure enters the rotation. Previous works assign each node a rotation from spectral coordinates, so the rotary factor between two nodes depends only on their endpoints and cannot distinguish the routes connecting them. We introduce \textit{Analytic-Walk Rotary Positional Encodings} (AW-RoPE), which place the rotations on edges and sum the transported features over all walks, so contributions along different routes can reinforce or cancel. An exact variant evaluates the complete sum by a differentiable linear solve, and a sparse variant truncates it at a finite depth. We prove forward and parameter-derivative truncation bounds at fixed inputs and parameters. Both variants act on projected queries and keys, and the sparse recurrence also augments message-passing networks. Across five synthetic tasks both variants reduce nRMSE by $15$--$58\%$ relative to the strongest baseline, and on real superpixel, peptide and OGB benchmarks the sparse recurrence attains the best mean on every dataset with Performer kernels and on twelve of thirteen datasets with GIN. Analysis shows that AW-RoPE can distinguish routes whose only cue is how two endpoints are connected, while node-wise rotary encodings cannot.
