arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.03227cs.CV
收藏

Spiral RoPE: 在二维平面上旋转你的旋转位置嵌入

Spiral RoPE: Rotate Your Rotary Positional Embeddings in the 2D Plane

University of California, Berkeley(加州大学伯克利分校);University of California, Santa Cruz(加州大学圣克ruz分校);John Hopkins University(约翰霍普金斯大学)

Haoyu Liu, Sucheng Ren, Tingyu Zhu, Peng Wang, Cihang Xie, Alan Yuille, Zeyu Zheng, Feng Wang

AI总结:

Spiral RoPE通过多方向位置编码改进视觉变换器在分类、分割和生成任务中的性能。

摘要:

旋转位置嵌入(RoPE)是由于其能够编码相对位置并支持长度外推而在大语言模型中成为事实上的位置编码。当适应到视觉变换器时,标准的轴向公式将二维空间位置分解为水平和垂直组件,隐含地限制了位置编码到轴对齐方向。我们识别出这个方向限制是标准轴向2D RoPE的根本限制,这阻碍了对自然图像中自然存在的斜向空间关系的建模。为了克服这一限制,我们提出了Spiral RoPE,一种简单而有效的扩展,通过将嵌入通道分成多个与均匀分布方向相关的组,实现了多方向的位置编码。每个组根据补丁位置在其相应方向上的投影进行旋转,从而允许在水平和垂直轴之外编码空间关系。在包括分类、分割和生成在内的广泛视觉任务中,Spiral RoPE始终提高了性能。对注意力图的定性分析进一步显示,Spiral RoPE在语义相关对象上表现出更集中的激活,并且更好地尊重局部物体边界,突显了多方向位置编码在视觉变换器中的重要性。

英文摘要:

Rotary Position Embedding (RoPE) is the de facto positional encoding in large language models due to its ability to encode relative positions and support length extrapolation. When adapted to vision transformers, the standard axial formulation decomposes two-dimensional spatial positions into horizontal and vertical components, implicitly restricting positional encoding to axis-aligned directions. We identify this directional constraint as a fundamental limitation of the standard axial 2D RoPE, which hinders the modeling of oblique spatial relationships that naturally exist in natural images. To overcome this limitation, we propose Spiral RoPE, a simple yet effective extension that enables multi-directional positional encoding by partitioning embedding channels into multiple groups associated with uniformly distributed directions. Each group is rotated according to the projection of the patch position onto its corresponding direction, allowing spatial relationships to be encoded beyond the horizontal and vertical axes. Across a wide range of vision tasks including classification, segmentation, and generation, Spiral RoPE consistently improves performance. Qualitative analysis of attention maps further show that Spiral RoPE exhibits more concentrated activations on semantically relevant objects and better respects local object boundaries, highlighting the importance of multi-directional positional encoding in vision transformers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑