arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.03410cs.CV
收藏

$R^{2}$Former:用于位置识别的统一 $R$etrieval 和 $R$eranking Transformer

$R^{2}$Former: Unified $R$etrieval and $R$eranking Transformer for Place Recognition

ByteDance(字节跳动);University of Central Florida(中佛罗里达大学)

Sijie Zhu, Linjie Yang, Chen Chen, Mubarak Shah, Xiaohui Shen, Heng Wang

AI总结:

提出统一的位置识别框架 $R^{2}$Former,利用新型 transformer 同时处理检索与重排序,结合特征相关性、注意力值和坐标进行判断,在主要 VPR 数据集上以更少的推理时间和内存消耗显著超越现有方法。

摘要:

视觉位置识别(VPR)通过将查询图像与参考数据库中的图像进行匹配来估计其位置。传统方法通常采用聚合的 CNN 特征进行全局检索,并使用基于 RANSAC 的几何验证进行重排序。然而,RANSAC 仅利用了几何信息,而忽略了其他可能对重排序有用的信息,例如局部特征相关性和注意力值。在本文中,我们提出了一个统一的位置识别框架,使用名为 $R^{2}$Former 的新型 transformer 模型同时处理检索和重排序。所提出的重排序模块考虑了特征相关性、注意力值和 xy 坐标,并学习判断图像对是否来自同一位置。整个流程是端到端可训练的,并且该重排序模块也可以作为通用组件应用于其他 CNN 或 transformer 主干网络。值得注意的是,$R^{2}$Former 在主要 VPR 数据集上显著优于现有最先进的方法,且推理时间和内存消耗要少得多。它还在留出的 MSLS 挑战集上达到了最先进的性能,可以作为现实世界大规模应用的简单而强大的解决方案。实验还表明,在局部匹配方面,vision transformer tokens 与 CNN 局部特征相当,有时甚至更好。代码已发布于 https://github.com/Jeff-Zilience/R2Former。

英文摘要:

Visual Place Recognition (VPR) estimates the location of query images by matching them with images in a reference database. Conventional methods generally adopt aggregated CNN features for global retrieval and RANSAC-based geometric verification for reranking. However, RANSAC only employs geometric information but ignores other possible information that could be useful for reranking, e.g. local feature correlations, and attention values. In this paper, we propose a unified place recognition framework that handles both retrieval and reranking with a novel transformer model, named $R^{2}$Former. The proposed reranking module takes feature correlation, attention value, and xy coordinates into account, and learns to determine whether the image pair is from the same location. The whole pipeline is end-to-end trainable and the reranking module alone can also be adopted on other CNN or transformer backbones as a generic component. Remarkably, $R^{2}$Former significantly outperforms state-of-the-art methods on major VPR datasets with much less inference time and memory consumption. It also achieves the state-of-the-art on the hold-out MSLS challenge set and could serve as a simple yet strong solution for real-world large-scale applications. Experiments also show vision transformer tokens are comparable and sometimes better than CNN local features on local matching. The code is released at https://github.com/Jeff-Zilence/R2Former.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑