DPPE：重新思考基于相机的可缩放多视图Transformer的位置编码
DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers
AI总结：
针对多视图Transformer中基于相机参数的位置编码导致训练后期性能停滞的问题，提出解耦位姿位置编码（DPPE），将旋转和平移显式分离，实现稳定长训练并提升外推泛化能力。
摘要：
Transformer显著的扩展性已将其应用扩展到3D计算机视觉，其中相机感知的位置编码对于在多视图几何中提供空间线索至关重要。最近的研究建立了使用相机参数（如外参或投影矩阵）作为注意力机制中查询、键和值向量的相对位置编码的实践。然而，当使用基于相机的位置编码扩展新视图合成（NVS）模型的训练方案时，我们观察到一个重要问题：模型性能在训练后期停滞。在本文中，我们研究了扩展时性能瓶颈的原因，并证明了将位置编码给出的旋转和平移存储在值向量的相同维度会导致其独立识别的不确定性，从而阻碍训练的可扩展性。为了解决这个问题，我们提出了解耦位姿位置编码（DPPE），一种新颖的基于相机的位置编码，它显式地解耦了旋转和平移。在NVS任务上的广泛评估表明，即使在扩展的训练设置中，DPPE也能实现稳定的长期训练。此外，它在外推设置（如处理增加视角数量和放大场景）中表现出优越的泛化性能。
英文摘要：
The remarkable scalability of Transformers has expanded their application to 3D computer vision, where camera-aware positional encoding is crucial for providing spatial cues in multi-view geometry. Recent advancements have established the practice of using camera parameters -- such as extrinsics or projection matrices -- as relative positional encoding into the query, key, and value vectors of the attention mechanism. However, when scaling up the training recipe of novel view synthesis (NVS) models with the camera-based positional encoding, we observe a significant issue: model performance stagnates in the late stages of training. In this paper, we investigate the cause of the performance bottleneck when scaling up and demonstrate that storing rotation and translation given by the positional encoding in the same dimensions of the value vector causes indeterminacy in their independent identification, hindering training scalability. To address this, we propose Decoupled Pose Positional Encoding (DPPE), a novel camera-based positional encoding that explicitly decouples rotation and translation. Extensive evaluations on NVS tasks demonstrate that DPPE enables stable long-term training even in scaled-up training setup. Furthermore, it exhibits superior generalization performance in extrapolation settings, such as handling an increased number of viewpoints and zoom-in scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
