arXiv 2510.027282025-11-06cs.RO

小米 EV-AD VLA 团队：面向跨模态无人机导航的标题引导检索系统——IROS 2025 RoboSense 挑战赛第 4 赛道技术报告

Team Xiaomi EV-AD VLA: Caption-Guided Retrieval System for Cross-Modal Drone Navigation -- Technical Report for IROS 2025 RoboSense Challenge Track 4

Tsinghua University（清华大学）；Xiaomi EV（小米电动车）；Georgia Institute of Technology（佐治亚理工学院）；National University of Singapore（新加坡国立大学）；The Chinese University of Hong Kong（香港中文大学）；Renmin University of China（中国人民大学）

Lingfeng Zhang, Erjia Xiao, Yuchen Zhang, Haoxiang Fu, Ruibin Hu, Yanbiao Ma, Wenbo Ding, Long Chen, Hangjun Ye, Xiaoshuai Hao

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对跨模态无人机导航中的文本-图像细粒度匹配难题，本文提出标题引导检索系统 CGRS，通过 VLM 生成候选图像标题并进行多模态重排序，在挑战赛中所有召回指标一致提升 5%，最终获得 TOP-2。

摘要：

跨模态无人机导航在机器人学中仍是一项具有挑战性的任务，需要基于自然语言描述从大规模数据库中高效检索相关图像。RoboSense 2025 第 4 赛道挑战赛针对这一挑战，聚焦于跨多个平台（无人机、卫星和地面相机）的鲁棒、自然语言引导的跨视角图像检索。当前的基线方法虽然在初始检索方面有效，但在文本查询与视觉内容之间实现细粒度语义匹配方面往往存在困难，尤其是在复杂的航拍场景中。为解决这一挑战，我们提出了一种两阶段检索精化方法：标题引导检索系统（CGRS），通过智能重排序增强基线粗排序。我们的方法首先利用基线模型为每个查询获得初始的前 20 个最相关图像的粗排序。然后，我们使用视觉语言模型（VLM）为这些候选图像生成详细的标题，捕捉其视觉内容的丰富语义描述。这些生成的标题随后被用于多模态相似度计算框架中，对原始文本查询进行细粒度重排序，从而在视觉内容与自然语言描述之间有效构建语义桥梁。我们的方法在基线上取得了显著提升，在所有关键指标（Recall@1、Recall@5 和 Recall@10）上均实现了 5% 的一致提升。我们的方法在挑战赛中获得 TOP-2，证明了我们的语义精化策略在真实机器人导航场景中的实用价值。

英文摘要：

Cross-modal drone navigation remains a challenging task in robotics, requiring efficient retrieval of relevant images from large-scale databases based on natural language descriptions. The RoboSense 2025 Track 4 challenge addresses this challenge, focusing on robust, natural language-guided cross-view image retrieval across multiple platforms (drones, satellites, and ground cameras). Current baseline methods, while effective for initial retrieval, often struggle to achieve fine-grained semantic matching between text queries and visual content, especially in complex aerial scenes. To address this challenge, we propose a two-stage retrieval refinement method: Caption-Guided Retrieval System (CGRS) that enhances the baseline coarse ranking through intelligent reranking. Our method first leverages a baseline model to obtain an initial coarse ranking of the top 20 most relevant images for each query. We then use Vision-Language-Model (VLM) to generate detailed captions for these candidate images, capturing rich semantic descriptions of their visual content. These generated captions are then used in a multimodal similarity computation framework to perform fine-grained reranking of the original text query, effectively building a semantic bridge between the visual content and natural language descriptions. Our approach significantly improves upon the baseline, achieving a consistent 5\% improvement across all key metrics (Recall@1, Recall@5, and Recall@10). Our approach win TOP-2 in the challenge, demonstrating the practical value of our semantic refinement strategy in real-world robotic navigation scenarios.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。