arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2406.05773cs.CV

基于几何一致预训练的可扩展且可泛化的对应关系修剪

Scalable and Generalizable Correspondence Pruning via Geometry-Consistent Pre-training

Wuhan University(武汉大学);Wenzhou University(温州大学);Nanjing University(南京大学);Xiaomi Corporation, Xiaomi Automobile Co., Ltd.(小米集团,小米汽车有限公司);Tongji University(同济大学)

Tangfei Liao, Xiaoqin Zhang, Tao Wang, Hao Ye, Min Li, Guobao Xiao, Mang Ye

AI总结:

本文提出几何一致预训练方法,通过掩码内点重建任务和双流编码器提升对应关系的鲁棒性和泛化能力,在相机姿态估计、视觉定位和3D配准任务中取得显著性能提升。

摘要:

两视图对应关系修剪旨在识别用于相机姿态估计的可靠对应关系,是许多3D视觉任务的基础步骤。现有方法依赖几何一致性在众多假对应关系(外点)中寻找真正的对应关系(内点)。在这一学习范式中,外点严重干扰内点的表示学习,导致模型既不鲁棒也不可泛化。为解决此问题,我们提出一种几何一致预训练范式,以塑造可扩展且可泛化的表示,不受外点干扰。该范式具有两个吸引人的特性。1)实现几何一致预训练。我们引入掩码内点重建作为预训练任务,并开发基于掩码自动编码器的简单而有效的预训练框架。具体而言,由于对应关系的不规则和无序性质,缺乏显式位置信息,我们采用双分支结构,分别重建两幅图像的关键点。这使得能够间接重建4D对应关系,其中配对图像的关键点提供位置提示。2)统一对应关系编码器。我们提出一个简单的双流编码器,内置共识交互,提供统一、可扩展的架构,增强表示学习。大量实验表明,我们的方法GeneralPruner在各种下游任务中均在鲁棒性和泛化能力方面优于现有最先进方法。具体而言,我们的方法在相机姿态估计、视觉定位和3D配准任务中分别实现了10.76%、11.84%和8.65%的性能提升。

英文摘要:

Two-view correspondence pruning aims to identify reliable correspondences for camera pose estimation, serving as a fundamental step in many 3D vision tasks. Existing methods rely on geometric consistency to seek true correspondences (inliers) from numerous false correspondences (outliers). In this learning paradigm, outliers severely affect the representation learning of inliers, resulting in models that are neither robust nor generalizable. To address this issue, we propose a geometry-consistent pre-training paradigm that sculpts scalable and generalizable representations free from outlier interference. The paradigm features two appealing properties. 1) Implementation of geometry-consistent pre-training. We introduce masked inlier reconstruction as a pretext task and develop a simple yet effective pre-training framework based on a masked autoencoder. Specifically, due to the irregular and unordered nature of correspondences, which lack explicit positional information, we adopt a dual-branch structure that separately reconstructs the keypoints of two images. This enables indirect reconstruction of 4D correspondences, where keypoints from the paired image provide positional prompts. 2) Unified correspondence encoder. We propose a simple dual-stream encoder with built-in consensus interaction, providing a unified, extensible architecture that enhances representation learning. Extensive experiments demonstrate that our method, GeneralPruner, consistently outperforms state-of-the-art approaches in terms of robustness and generalization across various downstream tasks. Specifically, our method achieves 10.76%, 11.84%, and 8.65% performance gains in camera pose estimation, visual localization, and 3D registration, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑