arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.08359cs.CV
收藏

学习为视觉定位生成半密集对应关系

Learning to Produce Semi-dense Correspondences for Visual Localization

School of Computing, KAIST(韩国科学技术院计算机学院);College of AI Convergence, Dongguk University(东国大学人工智能融合学院)

Khang Truong Giang, Soohwan Song, Sungho Jo

AI总结:

针对夜间、恶劣天气等苛刻条件下的视觉定位问题,提出一种基于密集关键点匹配提取半密集2D-3D匹配点的新颖定位方法,利用点推理网络推断未观测关键点的3D坐标,显著提升了相机位姿估计精度并在大规模基准上取得竞争力表现。

摘要:

本研究致力于解决在夜间场景、恶劣天气和季节变化等苛刻条件下进行视觉定位的挑战。尽管许多先前的研究致力于提升图像匹配性能,以实现图像间可靠的密集关键点匹配,但现有方法通常严重依赖重建3D模型上的预定义特征点。因此,它们在匹配过程中往往会忽略未被观测到的关键点。这导致密集关键点匹配未能被充分利用,进而导致精度显著下降,特别是在含噪场景中。为了解决这一问题,我们提出了一种新颖的定位方法,该方法基于密集关键点匹配来提取可靠的半密集2D-3D匹配点。该方法利用点推理网络将半密集2D关键点回归为3D场景坐标。该网络同时利用几何和视觉线索,从已观测关键点中有效地推断出未观测关键点的3D坐标。丰富的匹配信息显著提升了相机位姿估计的精度,即使在涉及含噪或稀疏3D模型的场景中也是如此。综合评估表明,所提方法在具有挑战性的场景中优于其他方法,并在大规模视觉定位基准上取得了具有竞争力的结果。代码将会公开。

英文摘要:

This study addresses the challenge of performing visual localization in demanding conditions such as night-time scenarios, adverse weather, and seasonal changes. While many prior studies have focused on improving image-matching performance to facilitate reliable dense keypoint matching between images, existing methods often heavily rely on predefined feature points on a reconstructed 3D model. Consequently, they tend to overlook unobserved keypoints during the matching process. Therefore, dense keypoint matches are not fully exploited, leading to a notable reduction in accuracy, particularly in noisy scenes. To tackle this issue, we propose a novel localization method that extracts reliable semi-dense 2D-3D matching points based on dense keypoint matches. This approach involves regressing semi-dense 2D keypoints into 3D scene coordinates using a point inference network. The network utilizes both geometric and visual cues to effectively infer 3D coordinates for unobserved keypoints from the observed ones. The abundance of matching information significantly enhances the accuracy of camera pose estimation, even in scenarios involving noisy or sparse 3D models. Comprehensive evaluations demonstrate that the proposed method outperforms other methods in challenging scenes and achieves competitive results in large-scale visual localization benchmarks. The code will be available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑