arXiv 2512.190212025-12-22cs.CVcs.RO

VLNVerse: 一个用于视觉语言导航的基准，具备多用途、具身体验、逼真模拟和评估

VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluation

Adelaide University（阿德莱德大学）；Responsible AI Research Centre, Australian Institute for Machine Learning（负责任人工智能研究中心、澳大利亚机器学习研究所）；Tongji University（同济大学）；ManyCore；Zhejiang University（浙江大学）；Shanghai AI Lab（上海人工智能实验室）；CSIRO Data61

Sihao Lin, Zerui Li, Xunyi Zhao, Gengze Zhou, Liuyi Wang, Rong Wei, Rui Tang, Juncheng Li, Hanqing Wang, Jiangmiao Pang, Anton van den Hengel, Jiajun Liu, Qi Wu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

VLNVerse是一个大规模、可扩展的视觉语言导航基准，通过多用途、具身体验和逼真模拟提升导航任务的泛化能力与研究效率。

摘要：

尽管在视觉语言导航（VLN）方面取得了显著进展，现有的基准仍然局限于固定、小规模的数据集和简单的物理模拟。这些不足限制了基准在模拟到现实泛化方面的洞察力，并造成了显著的研究空白。此外，任务碎片化阻碍了该领域的统一/共享进步，而有限的数据规模无法满足现代基于大语言模型（LLM）的预训练需求。为了克服这些限制，我们引入了VLNVerse：一个新规模大、可扩展的基准，用于多用途、具身体验、逼真模拟和评估。VLNVerse重新定义了VLN为一个可扩展的全栈具身体验AI问题。其多用途性质将之前碎片化的任务统一到一个框架中，并为研究人员提供了可扩展的工具包。其具身体验设计超越了不具身体验和传送的“幽灵”代理，支持在由强大物理引擎驱动的逼真模拟中的完整运动学。我们利用VLNVerse的规模和多样性，对现有方法进行全面评估，从经典模型到基于LLM的代理。我们还提出了一种新的统一多任务模型，能够解决基准中的所有任务。VLNVerse旨在缩小模拟导航与现实泛化之间的差距，为社区提供一个重要的工具，推动研究向可扩展、通用的具身体验移动代理发展。

英文摘要：

Despite remarkable progress in Vision-Language Navigation (VLN), existing benchmarks remain confined to fixed, small-scale datasets with naive physical simulation. These shortcomings limit the insight that the benchmarks provide into sim-to-real generalization, and create a significant research gap. Furthermore, task fragmentation prevents unified/shared progress in the area, while limited data scales fail to meet the demands of modern LLM-based pretraining. To overcome these limitations, we introduce VLNVerse: a new large-scale, extensible benchmark designed for Versatile, Embodied, Realistic Simulation, and Evaluation. VLNVerse redefines VLN as a scalable, full-stack embodied AI problem. Its Versatile nature unifies previously fragmented tasks into a single framework and provides an extensible toolkit for researchers. Its Embodied design moves beyond intangible and teleporting "ghost" agents that support full-kinematics in a Realistic Simulation powered by a robust physics engine. We leverage the scale and diversity of VLNVerse to conduct a comprehensive Evaluation of existing methods, from classic models to MLLM-based agents. We also propose a novel unified multi-task model capable of addressing all tasks within the benchmark. VLNVerse aims to narrow the gap between simulated navigation and real-world generalization, providing the community with a vital tool to boost research towards scalable, general-purpose embodied locomotion agents.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。