arXiv 2510.133102026-07-23cs.CV

InstantSfM: 向深度学习时代迈进的GPU原生SfM

InstantSfM: Towards GPU-Native SfM for the Deep Learning Era

Jiankun Zhong, Zitong Zhan, Quankai Gao, Ziyu Chen, Haozhe Lou, Jiageng Mao, Ulrich Neumann, Chen Wang, Yue Wang

AI总结：

InstantSfM通过深度约束的雅可比结构实现GPU原生SfM，解决尺度模糊问题，提升大规模场景下的效率和精度。

摘要：

结构从运动（SfM）是一种从多视角影像中恢复相机姿态和场景结构的基本技术，是3D重建和现代神经场景表示（如3D高斯散射）等应用中的关键上游组件。然而，大多数成熟的SfM系统仍以CPU为中心，基于传统的优化工具链，与现代基于GPU的、学习驱动的管道存在日益增长的不匹配，并限制了大规模场景的可扩展性。尽管最近在GPU加速的捆绑调整（BA）方面的进展展示了并行稀疏优化的潜力，但将这些技术扩展到构建完整的全局SfM系统仍然具有挑战性，因为存在未解决的度量尺度恢复和数值鲁棒性问题。在本文中，我们实现了一个完全基于GPU且与PyTorch兼容的全局SfM系统，名为InstantSfM，以无缝集成现代学习管道。InstantSfM通过深度约束的雅可比结构将度量深度先验直接嵌入到全局定位和BA中，从而在优化框架内解决尺度模糊问题。为确保数值稳定性，我们采用显式过滤欠约束变量的雅可比矩阵，以优化友好的GPU方式。在各种数据集上的广泛实验表明，InstantSfM在保持与已建立的经典管道和最近的学习方法相媲美的重建精度的同时，实现了最先进的效率，显示在大规模场景上比COLMAP快约40倍。

英文摘要：

Structure-from-Motion (SfM) is a fundamental technique for recovering camera poses and scene structure from multi-view imagery, serving as a critical upstream component for applications ranging from 3D reconstruction to modern neural scene representations such as 3D Gaussian Splatting. However, most mature SfM systems remain CPU-centric and built upon traditional optimization toolchains, creating a growing mismatch with modern GPU-based, learning-driven pipelines and limiting scalability in large-scale scenes. While recent advances in GPU-accelerated bundle adjustment (BA) have demonstrated the potential of parallel sparse optimization, extending these techniques to build a complete global SfM system remains challenging due to unresolved issues in metric scale recovery and numerical robustness. In this paper, we implement a fully GPU-based and PyTorch-compatible global SfM system, named InstantSfM, to integrate seamlessly with modern learning pipelines. InstantSfM embeds metric depth priors directly into both global positioning and BA through a depth-constrained Jacobian structure, thereby resolving scale ambiguity within the optimization framework. To ensure numerical stability, we employ explicit filtering of under-constrained variables for the Jacobian matrix in an optimized GPU-friendly manner. Extensive experiments on diverse datasets demonstrate that InstantSfM achieves state-of-the-art efficiency while maintaining reconstruction accuracy comparable to both established classical pipelines and recent learning-based methods, showing up to ${\sim40\times}$ speedup over COLMAP on large-scale scenes.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。