GROKE: 通过OpenStreetMap上的图推理进行无视觉导航指令评估
GROKE: Vision-Free Navigation Instruction Evaluation via Graph Reasoning on OpenStreetMap
AI总结：
GROKE提出一种基于OpenStreetMap的无视觉导航指令评估框架，通过图推理和拓扑导航提升评估精度与可扩展性。
摘要：
导航指令的评估仍然是视觉-语言导航（VLN）研究中的持续挑战。传统的基于参考的度量标准，如BLEU和ROUGE，无法捕捉空间指令的功能效用，特别是指令是否成功引导导航者到达目的地。尽管现有的VLN代理可以作为评估者，但它们依赖于高保真的视觉模拟器，引入了许可限制和计算成本，感知误差进一步混淆了语言质量评估。本文介绍了GROKE（基于OSM知识的图推理用于指令评估），一种无需视觉、无需训练的分层LLM框架，用于使用OpenStreetMap数据评估导航指令。通过系统的消融研究，我们证明结构化的JSON和文本格式的空间信息显著优于基于网格和视觉图的表示。我们的分层架构结合了子指令规划与拓扑图导航，将导航误差比Map2Seq数据集上的启发式和采样基线减少了68.5%。代理的执行成功率、轨迹保真度和决策模式作为给定OSM可见地标和拓扑的函数导航性的代理度量，建立了无视觉依赖的可扩展和可解释的评估范式。代码和数据可在https://anonymous.4open.science/r/groke获取。
英文摘要：
The evaluation of navigation instructions remains a persistent challenge in Vision-and-Language Navigation (VLN) research. Traditional reference-based metrics such as BLEU and ROUGE fail to capture the functional utility of spatial directives, specifically whether an instruction successfully guides a navigator to the intended destination. Although existing VLN agents could serve as evaluators, their reliance on high-fidelity visual simulators introduces licensing constraints and computational costs, and perception errors further confound linguistic quality assessment. This paper introduces GROKE(Graph-based Reasoning over OSM Knowledge for instruction Evaluation), a vision-free training-free hierarchical LLM-based framework for evaluating navigation instructions using OpenStreetMap data. Through systematic ablation studies, we demonstrate that structured JSON and textual formats for spatial information substantially outperform grid-based and visual graph representations. Our hierarchical architecture combines sub-instruction planning with topological graph navigation, reducing navigation error by 68.5% compared to heuristic and sampling baselines on the Map2Seq dataset. The agent's execution success, trajectory fidelity, and decision patterns serve as proxy metrics for functional navigability given OSM-visible landmarks and topology, establishing a scalable and interpretable evaluation paradigm without visual dependencies. Code and data are available at https://anonymous.4open.science/r/groke.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
