WorldRoPE：用于多视图注意力的概率性世界中心旋转位置嵌入
WorldRoPE: Probabilistic World-Centric Rotary Embedding for Multi-View Attention
AI总结：
针对多视图场景下RoPE无法建立跨视图对应关系的问题，提出概率性世界中心旋转位置嵌入WorldRoPE，通过体积高斯基元建模深度不确定性，在新视图合成等任务上验证了其有效性。
摘要：
旋转位置嵌入（RoPE）被广泛用于Transformer中编码相对位置。然而在多视图场景下，每个图像中独立定义了块坐标：同一个3D点在不同视图中可能有不同坐标，这使得这些坐标不足以建立跨视图对应关系。我们提出WorldRoPE，一种用于多视图注意力的概率性世界中心RoPE。在每一层，WorldRoPE将每个块令牌反向投影到共享世界坐标系中的一组体积高斯基元中。这些基元表示令牌的多个合理3D支撑，共同捕获替代深度假设、深度不确定性及其块视锥体的有限空间范围。当关键令牌的支撑被投影到查询视图时，它会在该关键令牌可能出现在查询图像中的位置上诱导一个分布。WorldRoPE以闭式形式在该分布上集成旋转算子，根据混合权重结合投影基元中心的旋转，并结合协方差相关的衰减。这使得注意力对查询视图中不确定的关键令牌位置的敏感性降低，同时无需光栅化或扭曲令牌特征。当支撑坍缩为一个点时，该公式退化为确定性RoPE。各层中，预测的几何结构引导跨视图注意力，而更新后的特征则细化下一层使用的几何结构。在新视图合成、深度估计和3D几何重建上的实验证明了WorldRoPE在多视图Transformer中的有效性。
英文摘要：
Rotary Positional Embedding (RoPE) is widely used to encode relative positions in Transformers. In multi-view settings, however, patch coordinates are defined independently in each image: the same 3D point can therefore have different coordinates across views, making them insufficient for establishing cross-view correspondences. We introduce WorldRoPE, a probabilistic world-centric RoPE for multi-view attention. At each layer, WorldRoPE back-projects each patch token into a set of volumetric Gaussian primitives in a shared world frame. These primitives represent multiple plausible 3D supports for a token, jointly capturing alternative depth hypotheses, depth uncertainty, and the finite spatial extent of its patch frustum. When a key token's support is projected into a query view, it induces a distribution over where that key may appear in the query image. WorldRoPE integrates the rotary operator over this distribution in closed form, combining rotations at the projected primitive centers according to their mixture weights with covariance-dependent attenuation. This makes attention less sensitive to uncertain key locations in the query view, without rasterizing or warping token features. The formulation reduces to deterministic RoPE when the support collapses to a point. Across layers, the predicted geometry guides cross-view attention, while the updated features refine the geometry used at the next layer. Experiments on novel view synthesis, depth estimation, and 3D geometry reconstruction demonstrate the effectiveness of WorldRoPE for multi-view Transformers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
