PanoSplatt3R：利用透视预训练实现通用无位姿宽基线全景重建
PanoSplatt3R: Leveraging Perspective Pretraining for Generalized Unposed Wide-Baseline Panorama Reconstruction
AI总结：
针对现有宽基线全景重建方法依赖精确位姿的问题，提出PanoSplatt3R，通过将透视域预训练适配到全景域、引入RoPE rolling建模全景周期性，实现无位姿下更优的新视角生成与深度估计。
摘要：
宽基线全景重建已成为一种高效且关键的方法，不仅可实现对周围3D环境的几何重建，还能生成高度真实、具沉浸感的新视角。现有方法虽在各类基准测试中表现出色，但大多依赖精确的位姿信息。在实际场景中，获取精确位姿往往需要额外的计算资源，且极易受噪声影响，这些局限阻碍了此类方法的广泛应用与实用性。本文提出PanoSplatt3R，一种无位姿的宽基线全景重建方法。我们将透视领域的基础重建预训练扩展并适配到全景领域，从而赋予模型强大的泛化能力。为实现平滑高效的域迁移，我们提出RoPE rolling（旋转位置编码滚动）机制，在不同注意力头的旋转位置编码中引入滚动坐标，在对RoPE机制改动极小的前提下，建模全景图像的水平周期性。综合实验表明，即便在无位姿信息的情况下，PanoSplatt3R仍显著优于当前最优方法，其优势在高质量新视角生成和深度估计精度两方面均有体现，展现出巨大的实际应用潜力。项目页面：https://npucvr.github.io/PanoSplatt3R
英文摘要：
Wide-baseline panorama reconstruction has emerged as a highly effective and pivotal approach for not only achieving geometric reconstruction of the surrounding 3D environment, but also generating highly realistic and immersive novel views. Although existing methods have shown remarkable performance across various benchmarks, they are predominantly reliant on accurate pose information. In real-world scenarios, the acquisition of precise pose often requires additional computational resources and is highly susceptible to noise. These limitations hinder the broad applicability and practicality of such methods. In this paper, we present PanoSplatt3R, an unposed wide-baseline panorama reconstruction method. We extend and adapt the foundational reconstruction pretrainings from the perspective domain to the panoramic domain, thus enabling powerful generalization capabilities. To ensure a seamless and efficient domain-transfer process, we introduce RoPE rolling that spans rolled coordinates in rotary positional embeddings across different attention heads, maintaining a minimal modification to RoPE's mechanism, while modeling the horizontal periodicity of panorama images. Comprehensive experiments demonstrate that PanoSplatt3R, even in the absence of pose information, significantly outperforms current state-of-the-art methods. This superiority is evident in both the generation of high-quality novel views and the accuracy of depth estimation, thereby showcasing its great potential for practical applications. Project page: https://npucvr.github.io/PanoSplatt3R
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
