AI 中文总结
针对零样本跨楼层视觉语言导航中语义规划与物理执行脱节的问题,提出PACE模块,通过可供性姿态条件化动作生成并引入失败感知偏好精炼,在R2R-CE和RxR-CE跨楼层子集上显著提升成功率,并验证了真实环境适用性。
AI 中文摘要
视觉与语言导航日益依赖通用语义规划器,然而在空间受限的过渡场景中,将正确的高层意图转化为可靠的物理执行仍然困难。到达楼梯、门口或狭窄通道并不保证能够通过;智能体必须识别出可执行的可供性姿态,并从累积的动作误差中恢复。我们提出PACE(偏好精炼的可供性条件执行),一个监督式局部执行模块,用于增强冻结的零样本语义规划器,实现可靠的跨楼层导航。PACE将过渡相关语义锚定到长期、以智能体为中心的可穿越可供性姿态上,并以此空间目标为条件生成短期动作,从而将语义目标与物理执行对齐。我们进一步通过失败感知的偏好精炼对PACE进行后训练,使用从轨迹中导出的对比正常或恢复行为与偏差放大行为的样本对,从而改进闭环修正。我们将PACE集成到六个开源零样本VLN导航器中,并在R2R-CE和RxR-CE的跨楼层子集上展示了一致的改进,平均成功率分别从16.35%提升至27.65%,从4.76%提升至12.06%。真实世界实验进一步证明了PACE在未见环境中的适用性,凸显了可穿越可供性在弥合语义意图与可靠具身行为之间的潜力。
英文摘要
Vision-and-language navigation increasingly relies on general-purpose semantic planners, yet translating correct high-level intent into reliable physical execution remains difficult in spatially constrained transitions. Reaching a staircase, doorway, or narrow passage does not ensure traversal; the agent must identify an executable affordance pose and recover from accumulated action errors. We propose PACE (Preference-refined Affordance-Conditioned Execution), a supervised local execution module that augments frozen zero-shot semantic planners for reliable cross-floor navigation. PACE grounds transition-related semantics into a long-horizon, agent-centric traversable affordance pose and conditions short-horizon action generation on this spatial target, thereby aligning semantic goals with physical execution. We further post-train PACE through failure-aware preference refinement using rollout-derived pairs that contrast normal or recovery behaviors with deviation-amplifying behaviors, thereby improving closed-loop correction. We integrate PACE into six open-source zero-shot VLN navigators and demonstrate consistent improvements on the cross-floor subsets of R2R-CE and RxR-CE, increasing the average success rate from 16.35% to 27.65% and from 4.76% to 12.06%, respectively. Real-world experiments further demonstrate PACE's applicability in unseen environments, highlighting the potential of traversable affordances to bridge semantic intent and reliable embodied behavior.