arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.14280cs.CV
收藏

RESfM:鲁棒的深度等变运动恢复结构

RESfM: Robust Deep Equivariant Structure from Motion

Weizmann Institute of Science(魏茨曼科学研究所);NVIDIA(英伟达)

Fadi Khatib, Yoni Kasten, Dror Moran, Meirav Galun, Ronen Basri

AI总结:

本文提出RESfM,通过引入多视图内点/离群点分类模块和鲁棒捆绑调整,解决了深度等变运动恢复结构方法对离群点敏感的问题,在含大量离群点的真实场景中取得最先进精度。

摘要:

多视图运动恢复结构(Structure from Motion)是一个基础且具有挑战性的计算机视觉问题。最近的一种基于深度学习的方法利用矩阵等变架构,从大规模图像集合中同时恢复相机位姿和三维场景结构。然而,该工作做了一个不现实的假设,即输入的点轨迹几乎不含离群点。本文提出了一种适合处理离群点的架构,通过增加一个尊重模型等变性的多视图内点/离群点分类模块,并利用鲁棒的捆绑调整步骤。实验表明,我们的方法可以成功应用于现实场景,包括大规模图像集合以及使用常见启发式方法提取的包含大量离群点的点轨迹,在几乎所有运行中达到了最先进的精度,优于现有的基于深度学习的方法,并与领先的经典(非深度)序列方法和全局方法相当。

英文摘要:

Multiview Structure from Motion is a fundamental and challenging computer vision problem. A recent deep-based approach utilized matrix equivariant architectures for simultaneous recovery of camera pose and 3D scene structure from large image collections. That work, however, made the unrealistic assumption that the point tracks given as input are almost clean of outliers. Here, we propose an architecture suited to dealing with outliers by adding a multiview inlier/outlier classification module that respects the model equivariance and by utilizing a robust bundle adjustment step. Experiments demonstrate that our method can be applied successfully in realistic settings that include large image collections and point tracks extracted with common heuristics that include many outliers, achieving state-of-the-art accuracies in almost all runs, superior to existing deep-based methods and on-par with leading classical (non-deep) sequential and global methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑