arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1912.09697cs.CV
收藏

DeepSFM:通过深度束调整运动恢复结构

DeepSFM: Structure From Motion Via Deep Bundle Adjustment

Fudan University(复旦大学);Google Research(谷歌研究院);Nuro, Inc(Nuro公司)

Xingkui Wei, Yinda Zhang, Zhuwen Li, Yanwei Fu, Xiangyang Xue

摘要:

运动恢复结构(SfM)是计算机视觉的一个重要问题,深度学习尚未很好地解决这个问题。其中一个有前途的趋势是应用明确的结构约束,如三维成本体积,到网络中。然而,现有的方法通常假设准确的摄像机位姿从GT或其他方法,这是不现实的,在实践中。在这项工作中,我们设计了一个物理驱动的架构,即DeepSFM,灵感来自传统的捆绑调整(BA),它包括两个基于成本体积的架构,分别用于深度和姿态估计,迭代运行,以改善两者。当与学习组件结合时,对深度(结构)和姿势(运动)的明确约束带来了传统BA和新兴深度学习技术的优点。在各种数据集上的大量实验表明,我们的模型在深度和姿态估计方面都达到了最先进的性能,并且对较少的输入和初始化中的噪声具有出色的鲁棒性。

英文摘要:

Structure from motion (SfM) is an essential computer vision problem which has not been well handled by deep learning. One of the promising trends is to apply explicit structural constraint, e.g. 3D cost volume, into the network. However, existing methods usually assume accurate camera poses either from GT or other methods, which is unrealistic in practice. In this work, we design a physical driven architecture, namely DeepSFM, inspired by traditional Bundle Adjustment (BA), which consists of two cost volume based architectures for depth and pose estimation respectively, iteratively running to improve both. The explicit constraints on both depth (structure) and pose (motion), when combined with the learning components, bring the merit from both traditional BA and emerging deep learning technology. Extensive experiments on various datasets show that our model achieves the state-of-the-art performance on both depth and pose estimation with superior robustness against less number of inputs and the noise in initialization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑