我们提出了 STRING:可分离平移不变位置编码。STRING 通过一个统一的理论框架扩展了旋转位置编码(Rotary Position Encodings),后者是最近提出并在大语言模型中广泛使用的算法。重要的是,STRING 仍然提供精确的平移不变性,包括任意维度的 token 坐标,同时保持较低的计算开销。这些特性在机器人领域尤为重要,因为高效的 3D token 表示是关键。我们将 STRING 集成到具有 RGB(-D) 输入(颜色加可选深度)的 Vision Transformer 中,显示出显著的增益,例如在开放词汇目标检测和机器人控制器方面。我们用严格的数学分析补充了实验,证明了我们方法的普适性。
英文摘要:
We introduce STRING: Separable Translationally Invariant Position Encodings. STRING extends Rotary Position Encodings, a recently proposed and widely used algorithm in large language models, via a unifying theoretical framework. Importantly, STRING still provides exact translation invariance, including token coordinates of arbitrary dimensionality, whilst maintaining a low computational footprint. These properties are especially important in robotics, where efficient 3D token representation is key. We integrate STRING into Vision Transformers with RGB(-D) inputs (color plus optional depth), showing substantial gains, e.g. in open-vocabulary object detection and for robotics controllers. We complement our experiments with a rigorous mathematical analysis, proving the universality of our methods.
Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Large Vision-Language Models
Huawei Noah's Ark Lab.(华为诺亚实验室);City University of Hong Kong.(香港城市大学);University of Sydney.(悉尼大学);State Key Lab of General AI, School of Intelligence Science and Technology, Peking University(北京大学人工智能国家重点实验室,智能科学与技术学院)
C^2ROPE: Causal Continuous Rotary Positional Encoding for 3D Large Multimodal-Models Reasoning
State Key Laboratory of Internet of Things for Smart City, University of Macau(物联网智能城市国家重点实验室,澳门大学);Department of Automation, Shanghai Jiaotong University(上海交通大学自动化系);Institute of Advanced Technology, University of Science and Technology of China(中国科学技术大学先进技术研究院);School of Computer Science and Technology, USTC(中国科学技术大学计算机科学与技术学院);School of Artificial Intelligence and Data Science, USTC(中国科学技术大学人工智能与数据科学学院)
VRoPE: Rotary Position Embedding for Video Large Language Models
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);School of Computer Science and Technology, Beijing Jiaotong University(北京交通大学计算机科学与技术学院);Basic Algorithm Center, Tencent(腾讯基础算法中心)
Technical Report for ICRA 2025 GOOSE 2D Semantic Segmentation Challenge: Leveraging Color Shift Correction, RoPE-Swin Backbone, and Quantile-based Label Denoising Strategy for Robust Outdoor Scene Understanding
Institute of Intelligent Systems(智能系统研究所);College of Artificial Intelligence, National Yang Ming Chiao Tung University(人工智能学院,阳明交通大学);Institute of Data Science, National Cheng Kung University(数据科学研究所,成功大学)
LinearARD: Linear-Memory Attention Distillation for RoPE Restoration
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Southwest University(西南大学);Dalian University of Technology(大连理工大学);University of Science and Technology Beijing(北京科技大学);University College London(伦敦大学学院)
Scalable Object Relation Encoding for Better 3D Spatial Reasoning in Large Language Models
Department of Computer Science and Engineering, Southern University of Science and Technology(南方科技大学计算机科学与工程系);Wangxuan Institute of Computer Technology, Peking University(北京大学王轩计算机技术研究所);State Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室)
Do traveling waves make good positional encodings?
Institute of Computer Science and Campus Institute Data Science, University of Göttingen(哥廷根大学计算机科学学院和校园数据科学研究所);Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany(马克斯·普朗克动态与自组织研究所)