基于图像的机器人地理定位：黑盒视觉语言模型是否已经足够？
Image-based Geo-localization for Robotics: Are Black-box Vision-Language Models there yet?
AI总结：
本文首次系统研究黑盒生成式视觉语言模型作为独立零样本地理定位系统的潜力，发现其在粗粒度定位上表现良好，但在细粒度定位上因现实变化而显著退化。
摘要：
视觉语言模型（VLM）的进步为涉及图像地理定位的机器人应用提供了令人兴奋的机会——即仅基于视觉数据识别地点地理坐标的问题。在机器人学中，这种能力对于绑架机器人问题的全局重定位阶段尤其相关，其中机器人必须在没有先验位置信息的情况下恢复其姿态。最近的工作集中在使用VLM作为地理定位的嵌入提取器。然而，最先进的VLM可能仅作为通过API访问的黑盒，并带有许多限制：无法访问训练数据、模型特征和梯度；无法重新训练；预测次数可能受API限制。最先进的VLM作为独立、零样本地理定位系统在行星尺度上使用单个文本提示的潜力尚未得到充分探索。为填补这一空白，本文首次系统研究（据我们所知）最先进的生成式VLM作为独立、零样本地理定位系统在具有现实约束的黑盒设置中的表现。我们考虑了三种主要场景进行深入研究：a) 固定文本提示；b) 语义等效的文本提示；c) 语义等效的查询图像。除了标准准确性外，我们引入模型一致性作为度量，以考虑生成式VLM的自回归和概率性质。我们的发现揭示，虽然VLM表现出强大的粗粒度定位和导航先验，但细粒度定位在现实变化下显著退化，凸显了在鲁棒、开放世界机器人导航系统中部署生成式VLM的可靠性挑战。
英文摘要：
The advances in Vision-Language models (VLMs) offer exciting opportunities for robotic applications involving image geo-localization - the problem of identifying the geo-coordinates of a place based on visual data only. In robotics, such capabilities are particularly relevant to the global re-localization stage of the kidnapped robot problem, where a robot must recover its pose without prior knowledge of its location. Recent work has focused on using a VLM as embedding extractor for geo-localization. However, the most sophisticated VLMs may only be available as black boxes that are accessible through an API, and come with a number of limitations: there is no access to training data, model features and gradients; retraining is not possible; and the number of predictions may be limited by the API. The potential of state-of-the-art VLMs as a stand-alone, zero-shot geo-localization systems at planet scale using a single text-based prompt is largely unexplored. To bridge this gap, this paper undertakes the first systematic study, to the best of our knowledge, to investigate state-of-the-art generative VLMs as stand-alone, zero-shot geo-localization systems in a black-box setting with realistic constraints. We consider three main scenarios for this thorough investigation: a) fixed text-based prompt; b) semantically-equivalent text-based prompts; and c) semantically-equivalent query images. Beyond standard accuracy, we introduce model consistency as a metric to account for the auto-regressive and probabilistic nature of generative VLMs. Our findings reveal that while VLMs demonstrate strong coarse-level localization and navigation priors, fine-grained localization degrades significantly under realistic variations, highlighting reliability challenges for deploying generative VLMs in robust, open-world robotic navigation systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
