融合细粒度对齐标注的视觉-语言导航指令生成
Generating Vision-Language Navigation Instructions Incorporated Fine-Grained Alignment Annotations
AI总结：
针对 VLN 缺乏细粒度跨模态标注的问题，提出 FCA-NIG 自动构建 FCA-R2R 数据集，提升多个智能体的导航性能与泛化能力。
摘要：
视觉-语言导航（VLN）使智能体能够通过整合视觉感知和自然语言指令在环境中导航，但由于缺乏细粒度跨模态对齐标注而面临重大挑战。现有数据集主要关注全局指令-轨迹匹配，忽视了对准确导航动作决策至关重要的子指令级和实体级对齐。为解决这一局限，我们提出 FCA-NIG，这是一个自动构建带有双层细粒度跨模态标注的导航指令的生成式框架。在该框架中，增强轨迹首先被划分为子轨迹，随后经过基于 GLIP 的地标检测、精心设计的指令构建、基于 OFA-Speaker 的类 R2R 指令生成以及由 CLIP 驱动的实体选择，生成带有实体-地标标注的子指令-轨迹对。最后，这些子对被聚合以形成完整的指令-轨迹对。该框架生成了 FCA-R2R 数据集，这是首个具有精确子指令-子轨迹和实体-地标对齐的大规模增强数据集。大量实验表明，使用 FCA-R2R 训练显著提升了多种最先进 VLN 智能体的性能，包括 SF、EnvDrop、RecBERT 和 HAMT。融合子指令-轨迹对齐增强了智能体的状态感知和决策准确性，而实体-地标对齐进一步提升了导航性能和泛化能力。这些结果凸显了 FCA-NIG 在无需人工标注的情况下生成高质量、可扩展训练数据的有效性，推动了复杂导航任务中的细粒度跨模态学习。
英文摘要：
Vision-Language Navigation (VLN) enables intelligent agents to navigate environments by integrating visual perception and natural language instructions, yet faces significant challenges due to the scarcity of fine-grained cross-modal alignment annotations. Existing datasets primarily focus on global instruction-trajectory matching, neglecting sub-instruction-level and entity-level alignments critical for accurate navigation action decision-making. To address this limitation, we propose FCA-NIG, a generative framework that automatically constructs navigation instructions with dual-level fine-grained cross-modal annotations. In this framework, an augmented trajectory is first divided into sub-trajectories, which are then processed through GLIP-based landmark detection, crafted instruction construction, OFA-Speaker based R2R-like instruction generation, and CLIP-powered entity selection, generating sub-instruction-trajectory pairs with entity-landmark annotations. Finally, these sub-pairs are aggregated to form a complete instruction-trajectory pair. The framework generates the FCA-R2R dataset, the first large-scale augmentation dataset featuring precise sub-instruction-sub-trajectory and entity-landmark alignments. Extensive experiments demonstrate that training with FCA-R2R significantly improves the performance of multiple state-of-the-art VLN agents, including SF, EnvDrop, RecBERT, and HAMT. Incorporating sub-instruction-trajectory alignment enhances agents' state awareness and decision accuracy, while entity-landmark alignment further boosts navigation performance and generalization. These results highlight the effectiveness of FCA-NIG in generating high-quality, scalable training data without manual annotation, advancing fine-grained cross-modal learning in complex navigation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
