HaltNav: 基于轻量拓扑先验的反应式视觉停止用于鲁棒视觉-语言导航
HaltNav: Reactive Visual Halting over Lightweight Topological Priors for Robust Vision-Language Navigation
AI总结：
本文提出HaltNav框架，结合轻量拓扑先验与局部探索能力，通过反应式视觉停止机制提升视觉-语言导航的鲁棒性，实验表明其在复杂环境下的表现优于现有方法。
摘要：
视觉-语言导航（VLN）正从刚性逐步指令跟随转向开放词汇、目标导向的自主性。实现这一转变无需详尽路由提示，需使代理利用结构先验。尽管先前工作常假设计算密集型2D/3D度量地图，我们则利用轻量文本基的osmAG（OpenStreetMap Area Graph），一种易于获取和维护的楼层级拓扑表示。然而，仅凭全局规划先验地图在现实部署中易脆，因局部连通性变化（如关闭门或拥挤通道）导致执行失败。为此，我们提出HaltNav分层导航框架，将osmAG的稳健全局规划与VLN的局部探索和指令 grounding 能力结合。我们的方法包含基于MLLM的脑模块，可实现高层任务 grounding 和障碍物意识。基于osmAG，脑模块将全局路线转换为局部执行片段序列，为VLN执行器提供先验引导、目标导向的子指令。同时，它通过我们称为反应式视觉停止（RVH）的机制检测局部异常，中断局部控制循环，更新osmAG并触发重新规划以安排可行绕行。为高效训练此停止能力，我们引入数据合成管道，利用生成模型注入真实障碍物到可通行场景中，显著丰富难负样本。大量实验表明，我们的分层框架在无繁琐语言指令的情况下优于多个基线方法，并显著提高在环境变化下的长周期视觉-语言导航鲁棒性。
英文摘要：
Vision-and-Language Navigation (VLN) is shifting from rigid, step-by-step instruction following toward open-vocabulary, goal-oriented autonomy. Achieving this transition without exhaustive routing prompts requires agents to leverage structural priors. While prior work often assumes computationally heavy 2D/3D metric maps, we instead exploit a lightweight, text-based osmAG (OpenStreetMap Area Graph), a floorplan-level topological representation that is easy to obtain and maintain. However, global planning over a prior map alone is brittle in real-world deployments, where local connectivity can change (e.g., closed doors or crowded passages), leading to execution-time failures. To address this gap, we propose a hierarchical navigation framework HaltNav that couples the robust global planning of osmAG with the local exploration and instruction-grounding capability of VLN. Our approach features an MLLM-based brain module, which is capable of high-level task grounding and obstruction awareness. Conditioned on osmAG, the brain converts the global route into a sequence of localized execution snippets, providing the VLN executor with prior-grounded, goal-centric sub-instructions. Meanwhile, it detects local anomalies via a mechanism we term Reactive Visual Halting (RVH), which interrupts the local control loop, updates osmAG by invalidating the corresponding topology, and triggers replanning to orchestrate a viable detour. To train this halting capability efficiently, we introduce a data synthesis pipeline that leverages generative models to inject realistic obstacles into otherwise navigable scenes, substantially enriching hard negative samples. Extensive experiments demonstrate that our hierarchical framework outperforms several baseline methods without tedious language instructions, and significantly improves robustness for long-horizon vision-language navigation under environmental changes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
