具有旋转位置嵌入的自注意力动态：球面上的扭曲状态和明确的共识率
Self-Attention Dynamics with Rotary Position Embeddings: Twisted States and Explicit Consensus Rates on the Sphere
- Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences（中国科学院西安光学精密机械研究所）
- University of Chinese Academy of Sciences（中国科学院大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
研究旋转位置嵌入对自注意力动态的影响，通过研究查询和键旋转时值保持在单位球面上的连续时间动态，推导出相关谱，得出全局和局部的一些结论以及RoPE的分数扁平化扭曲分支情况，并用多种计算交叉检查结果。
AI中文摘要：
旋转位置嵌入（RoPE）通过与位置相关的旋转来修改注意力分数，但普通的球面自注意力模型未捕捉到其对归一化令牌动态的影响。我们研究了查询和键旋转而值保持在单位球面上时得到的连续时间动态。得到的注意力核是可逆的，并且有一个尖锐的均匀softmax下限。在共振单频环上，我们推导出精确的贝塞尔混叠谱。全局上，封闭半球是不变的，成对非钝角配置和严格开放半圆会收缩。RoPE还选择了一个明确的分数扁平化扭曲分支，在多维中，局部共识差距可能非单调地依赖于跨频率平面的能量分配。通过多种计算交叉检查了定理边界和报告的常数。
英文摘要：
Rotary position embeddings (RoPE) modify attention scores through position-dependent rotations, but their effect on normalized token dynamics is not captured by the vanilla spherical self-attention model. We study the continuous-time dynamics obtained when queries and keys are rotated while values remain on the unit sphere. The resulting attention kernel is reversible and admits a sharp uniform softmax floor, yet the natural RoPE interaction energy has derivatives of both signs within one fixed nontrivial system. Every consensus state remains an equilibrium, and its transverse linearization is a reversible Markov operator whose kernel depends on the consensus point through its energy across RoPE planes. On a resonant single-frequency ring we derive an exact Bessel-aliasing spectrum, including non-coprime frequencies and the correct fixed-ring large-$β$ asymptotics. Globally, closed hemispheres are invariant, while pairwise non-obtuse configurations and strict open semicircles contract with explicit half-angle and single-point tail bounds. These regional estimates instantiate a kernel-generic positivity principle with the sharp RoPE softmax floor. RoPE also selects an explicit score-flattening twisted branch; the generic resonant family is non-hyperbolic and linearly unstable, whereas an odd antipodal family becomes a hyperbolic saddle after quotienting global rotation. In multiple dimensions, the local consensus gap can depend non-monotonically on the allocation of energy across frequency planes, so no universal ordering by frequency is valid. Independent matrix, finite-difference, and nonlinear-flow computations cross-check the theorem boundaries and the reported constants.
