arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.14095cs.CVcs.ROeess.IV
收藏

PanoVPR:通过跨全景视图的滑动窗口迈向统一的透视到等距柱状视觉位置识别

PanoVPR: Towards Unified Perspective-to-Equirectangular Visual Place Recognition via Sliding Windows across the Panoramic View

Zhejiang University(浙江大学);Shanghai SUPREMIND Technology Co., Ltd(上海超擎人工智能科技有限公司);Hunan University(湖南大学)

Ze Shi, Hao Shi, Kailun Yang, Zhe Yin, Yining Lin, Kaiwei Wang

AI总结:

针对透视到等距柱状(P2E)视觉位置识别中硬裁剪导致的特征截断问题,提出统一框架PanoVPR,利用滑动窗口计算特征描述符并支持直接迁移P2P主干网络,在自建数据集上取得SOTA性能。

摘要:

视觉位置识别作为自动驾驶和机器人领域的关键技术,近年来受到了广泛关注。目前,两种主要方法是透视视图检索(P2P)范式和等距柱状图像检索(E2E)范式。然而,在实际场景中,更自然且合理的假设是用户仅使用消费级针孔相机获取查询透视图像,并在地图提供商提供的全景数据库图像中进行检索。为此,我们提出了PanoVPR,这是一个透视到等距柱状(P2E)视觉位置识别框架,它采用滑动窗口来消除由硬裁剪引起的特征截断。具体而言,PanoVPR在整个等距柱状图像上滑动窗口,并为每个窗口计算特征描述符,随后通过比较这些描述符来确定位置相似度。值得注意的是,我们的统一框架能够将P2P方法的主干网络直接进行迁移而无需任何修改,不仅支持CNNs,也支持Transformers。为了便于训练和评估,我们从Pitts250k数据集衍生出Pitts250k-P2E数据集,并构建了YQ360数据集,后者是首个由移动机器人平台采集的P2E视觉位置识别数据集,旨在更好地模拟真实世界的任务场景。大量实验表明,PanoVPR达到了最先进的性能,并在Pitts250k-P2E和YQ360数据集上分别取得了3.8%和8.0%的性能提升。代码和数据集将在 https://github.com/zafirshi/PanoVPR 公开。

英文摘要:

Visual place recognition has gained significant attention in recent years as a crucial technology in autonomous driving and robotics. Currently, the two main approaches are the perspective view retrieval (P2P) paradigm and the equirectangular image retrieval (E2E) paradigm. However, it is practical and natural to assume that users only have consumer-grade pinhole cameras to obtain query perspective images and retrieve them in panoramic database images from map providers. To address this, we propose \textit{PanoVPR}, a perspective-to-equirectangular (P2E) visual place recognition framework that employs sliding windows to eliminate feature truncation caused by hard cropping. Specifically, PanoVPR slides windows over the entire equirectangular image and computes feature descriptors for each window, which are then compared to determine place similarity. Notably, our unified framework enables direct transfer of the backbone from P2P methods without any modification, supporting not only CNNs but also Transformers. To facilitate training and evaluation, we derive the Pitts250k-P2E dataset from the Pitts250k and establish YQ360, latter is the first P2E visual place recognition dataset collected by a mobile robot platform aiming to simulate real-world task scenarios better. Extensive experiments demonstrate that PanoVPR achieves state-of-the-art performance and obtains 3.8% and 8.0% performance gain on Pitts250k-P2E and YQ360 compared to the previous best method, respectively. Code and datasets will be publicly available at https://github.com/zafirshi/PanoVPR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑