arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.12351cs.ROcs.CV
收藏

LongNav-R1: 基于水平自适应的多轮强化学习用于长周期视觉语言导航

LongNav-R1: Horizon-Adaptive Multi-Turn RL for Long-Horizon VLA Navigation

University of Michigan(密歇根大学)

Yue Hu, Avery Xi, Qixin Xiao, Seth Isaacson, Henry X. Liu, Ram Vasudevan, Maani Ghaffari

AI总结:

LongNav-R1通过多轮强化学习提升长周期视觉语言导航性能,实现高效样本利用和多样化导航行为。

摘要:

本文开发了LongNav-R1,一种端到端的多轮强化学习(RL)框架,旨在优化视觉-语言-动作(VLA)模型以实现长周期导航。与现有单轮范式不同,LongNav-R1将导航决策过程重新表述为VLA策略与具身环境之间的连续多轮对话。这种多轮RL框架具有两个显著优势:i)它使代理能够推理历史交互的因果效应和连续未来结果;ii)它允许模型直接从在线交互中学习,生成多样化的轨迹并避免人类演示通常施加的行为刚性。此外,我们引入了水平自适应策略优化。该机制在优势估计过程中显式考虑变化的周期长度,从而在长序列上实现准确的时间信用分配。因此,代理能够发展出多样化的导航行为,并在长周期任务中抵抗崩溃。在物体导航基准测试中,实验验证了该框架的有效性:使用4,000条轨迹,LongNav-R1将Qwen3-VL-2B的成功率从64.3%提升至73.0%。这些结果展示了优越的样本效率,并显著优于最先进的方法。模型的通用性和鲁棒性还通过其在长周期真实世界导航设置中的零样本表现得到进一步验证。所有源代码将在发表时开源。

英文摘要:

This paper develops LongNav-R1, an end-to-end multi-turn reinforcement learning (RL) framework designed to optimize Visual-Language-Action (VLA) models for long-horizon navigation. Unlike existing single-turn paradigm, LongNav-R1 reformulates the navigation decision process as a continuous multi-turn conversation between the VLA policy and the embodied environment. This multi-turn RL framework offers two distinct advantages: i) it enables the agent to reason about the causal effects of historical interactions and sequential future outcomes; and ii) it allows the model to learn directly from online interactions, fostering diverse trajectory generation and avoiding the behavioral rigidity often imposed by human demonstrations. Furthermore, we introduce Horizon-Adaptive Policy Optimization. This mechanism explicitly accounts for varying horizon lengths during advantage estimation, facilitating accurate temporal credit assignment over extended sequences. Consequently, the agent develops diverse navigation behaviors and resists collapse during long-horizon tasks. Experiments on object navigation benchmarks validate the framework's efficacy: With 4,000 rollout trajectories, LongNav-R1 boosts the Qwen3-VL-2B success rate from 64.3% to 73.0%. These results demonstrate superior sample efficiency and significantly outperform state-of-the-art methods. The model's generalizability and robustness are further validated by its zero-shot performance in long-horizon real-world navigation settings. All source code is open-sourced at https://github.com/UMich-CURLY/LongNav-R1.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑