细粒度指令引导的图推理用于视觉-语言导航
Fine-Grained Instruction-Guided Graph Reasoning for Vision-and-Language Navigation
AI总结：
本文提出细粒度指令引导的图推理框架OIKG，通过解耦角度与视觉提示并增强空间表示，提升视觉-语言导航中指令理解和跨模态对齐能力。
摘要：
视觉-语言导航（VLN）要求一个具身代理通过自然语言指令在复杂环境中移动，需要准确对齐视觉观察和语言指导。尽管近年来取得了进展，但现有方法通常将视觉和方向提示以耦合的方式编码，并且在处理指令时没有显式提取导航关键语义，这往往导致不精确的空间推理和次优跨模态对齐。为了解决这些挑战，我们提出了一种细粒度指令引导的图推理框架（OIKG），在导航过程中增强空间表示和指令理解。具体来说，引入了一个观察图交互机制，以解耦角度和视觉提示，同时通过几何嵌入加强有向边表示，从而在导航图中实现更可靠的空间推理。此外，设计了一个细粒度指令引导模块，以显式提取和利用语言指令中的位置特定和对象中心信息，促进语言语义与可导航轨迹之间的更精确跨模态对齐。通过联合整合结构化的图推理与指令关键语义提示，所提出的方法显著提高了代理遵循复杂导航指令的能力。在R2R和RxR基准上的广泛实验表明，我们的方法在多个评估指标上均实现了最先进的性能，验证了细粒度指令引导的图推理在视觉-语言导航中的有效性。
英文摘要：
Vision and Language Navigation (VLN) requires an embodied agent to traverse complex environments by following natural language instructions, demanding accurate alignment between visual observations and linguistic guidance. To address these challenges, we propose a fine grained instruction guided graph reasoning framework (FIGR) that enhances both spatial representation and instruction understanding during navigation. Specifically, an observation graph interaction mechanism is introduced to disentangle angular and visual cues while strengthening directed edge representations through geometric embedding, enabling more reliable spatial reasoning within the navigation graph. The key detail guidance module is implemented as Adaptive Open Vocabulary Guidance (AOVG), where a contextual role parser dynamically identifies location, object, spatial relation, and other contextual cues. This design avoids exact string matching and supports previously unseen entities and compositional expressions. For multilingual instructions, a Multilingual Semantic Adapter (MSA) maps language-specific representations into a shared navigation-semantic space. By jointly integrating structured graph reasoning with instruction critical semantic cues, the proposed approach significantly improves the agent ability to follow complex navigation instructions. On the validation-unseen splits, FIGR achieves 67 SPL on R2R and 64.8 sDTW on RxR, exceeding SPENav by 1 percentage point in SPL and PRET by 2.4 points in sDTW, respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
