NAVCON：一个面向视觉与语言导航的认知启发与语言学基础语料库
NAVCON: A Cognitively Inspired and Linguistically Grounded Corpus for Vision and Language Navigation
AI总结：
本文提出了NAVCON，一个基于R2R和RxR的大规模视觉语言导航标注语料库，引入了四个核心导航概念及银标注生成算法，并通过人工评估和模型训练验证了其质量与实用性。
摘要：
我们提出了NAVCON，这是一个基于两个流行数据集（R2R和RxR）构建的大规模标注视觉语言导航（VLN）语料库。本文引入了四个核心的、认知驱动的和语言学基础的导航概念，以及一种算法，用于在导航指令中生成这些概念自然语言实现的规模化银标注。我们将标注的指令与智能体执行这些指令的视频片段配对。NAVCON包含约30,0000条指令的236,316个概念标注，以及2.7百万张对齐图像（来自约19,000条指令），展示了智能体执行指令时所见的内容。据我们所知，这是首个全面的导航概念资源。我们通过对NAVCON样本进行人类评估研究来评估银标注的质量。作为对该资源质量和有用性的进一步验证，我们训练了一个模型来检测未见指令中的导航概念及其语言实现。此外，我们展示了使用NAVCON的大规模银标注，GPT-4o的少样本学习在此任务上表现良好。
英文摘要：
We present NAVCON, a large-scale annotated Vision-Language Navigation (VLN) corpus built on top of two popular datasets (R2R and RxR). The paper introduces four core, cognitively motivated and linguistically grounded, navigation concepts and an algorithm for generating large-scale silver annotations of naturally occurring linguistic realizations of these concepts in navigation instructions. We pair the annotated instructions with video clips of an agent acting on these instructions. NAVCON contains 236, 316 concept annotations for approximately 30, 0000 instructions and 2.7 million aligned images (from approximately 19, 000 instructions) showing what the agent sees when executing an instruction. To our knowledge, this is the first comprehensive resource of navigation concepts. We evaluated the quality of the silver annotations by conducting human evaluation studies on NAVCON samples. As further validation of the quality and usefulness of the resource, we trained a model for detecting navigation concepts and their linguistic realizations in unseen instructions. Additionally, we show that few-shot learning with GPT-4o performs well on this task using large-scale silver annotations of NAVCON.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
