arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2505.03422cs.CVcs.RO
收藏

LiftFeat:3D几何感知的局部特征匹配

LiftFeat: 3D Geometry-Aware Local Feature Matching

School of Computer Science, Wuhan University(武汉大学计算机学院);SF Technology(SF科技);School of Computer Science, Central China Normal University(中央财经大学计算机学院);Institute for Infocomm Research, A*STAR(资讯研究院)

Yepeng Liu, Wenpeng Lai, Zhou Zhao, Yuxuan Xiong, Jinchi Zhu, Jun Cheng, Yongchao Xu

AI总结:

本文提出轻量级网络LiftFeat,通过融合单目深度估计得到的3D表面法线特征与2D描述子,提升极端条件下局部特征匹配的鲁棒性和判别力,并在多项任务中优于现有轻量级方法。

摘要:

鲁棒且高效的局部特征匹配在机器人领域的SLAM和视觉定位等应用中起着至关重要的作用。尽管已取得巨大进展,但在光照剧烈变化、低纹理区域或重复纹理等场景中,提取鲁棒且具有判别力的视觉特征仍然非常具有挑战性。本文提出了一种名为LiftFeat的新型轻量级网络,通过聚合3D几何特征来提升原始描述子的鲁棒性。具体而言,我们首先采用预训练的单目深度估计模型生成伪表面法线标签,以预测表面法线的形式监督3D几何特征的提取。然后,我们设计了一个3D几何感知的特征提升模块,将表面法线特征与原始2D描述子特征进行融合。融合此类3D几何特征增强了2D特征描述在极端条件下的判别能力。在相对位姿估计、单应性估计和视觉定位任务上的大量实验结果表明,我们的LiftFeat优于一些轻量级的当前最优方法。代码将发布在:https://github.com/lyp-deeplearning/LiftFeat。

英文摘要:

Robust and efficient local feature matching plays a crucial role in applications such as SLAM and visual localization for robotics. Despite great progress, it is still very challenging to extract robust and discriminative visual features in scenarios with drastic lighting changes, low texture areas, or repetitive patterns. In this paper, we propose a new lightweight network called \textit{LiftFeat}, which lifts the robustness of raw descriptor by aggregating 3D geometric feature. Specifically, we first adopt a pre-trained monocular depth estimation model to generate pseudo surface normal label, supervising the extraction of 3D geometric feature in terms of predicted surface normal. We then design a 3D geometry-aware feature lifting module to fuse surface normal feature with raw 2D descriptor feature. Integrating such 3D geometric feature enhances the discriminative ability of 2D feature description in extreme conditions. Extensive experimental results on relative pose estimation, homography estimation, and visual localization tasks, demonstrate that our LiftFeat outperforms some lightweight state-of-the-art methods. Code will be released at : https://github.com/lyp-deeplearning/LiftFeat.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑