arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.03114cs.CVcs.LGcs.RO
收藏

自我监督几何知觉

Self-supervised Geometric Perception

MIT(麻省理工学院);CMU(卡内基梅隆大学);Intel Labs(英特尔实验室)

Heng Yang, Wei Dong, Luca Carlone, Vladlen Koltun

摘要:

我们提出了自监督几何感知(SGP),这是第一个学习用于对应匹配的特征描述符的通用框架,而无需任何地面真实几何模型标签(例如,相机姿势、刚性变换)。我们的第一个贡献是将几何感知公式化为优化问题,该优化问题在给定大的视觉测量语料库的情况下联合优化特征描述符和几何模型(例如,图像、点云)。在这种优化公式下,我们展示了视觉研究的两个重要流,即鲁棒模型拟合和深度特征学习,对应于优化未知变量的一个块,同时固定另一个块。这种分析自然会导致我们的第二个贡献-SGP算法,执行交替最小化,以解决联合优化。SGP迭代地执行两个元算法:一个教师执行鲁棒的模型拟合给定的学习特征以生成几何伪标签,一个学生在伪标签的噪声监督下执行深度特征学习。作为第三个贡献,我们将SGP应用于大规模真实数据集上的两个感知问题,即MegaDepth上的相对相机姿态估计和3DMatch上的点云配准。我们证明,SGP实现了最先进的性能,与使用地面真实标签训练的监督预言机相当或更好。

英文摘要:

We present self-supervised geometric perception (SGP), the first general framework to learn a feature descriptor for correspondence matching without any ground-truth geometric model labels (e.g., camera poses, rigid transformations). Our first contribution is to formulate geometric perception as an optimization problem that jointly optimizes the feature descriptor and the geometric models given a large corpus of visual measurements (e.g., images, point clouds). Under this optimization formulation, we show that two important streams of research in vision, namely robust model fitting and deep feature learning, correspond to optimizing one block of the unknown variables while fixing the other block. This analysis naturally leads to our second contribution -- the SGP algorithm that performs alternating minimization to solve the joint optimization. SGP iteratively executes two meta-algorithms: a teacher that performs robust model fitting given learned features to generate geometric pseudo-labels, and a student that performs deep feature learning under noisy supervision of the pseudo-labels. As a third contribution, we apply SGP to two perception problems on large-scale real datasets, namely relative camera pose estimation on MegaDepth and point cloud registration on 3DMatch. We demonstrate that SGP achieves state-of-the-art performance that is on-par or superior to the supervised oracles trained using ground-truth labels.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑