arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.07028cs.AIcs.CLcs.CVcs.LGcs.RO
收藏

一步一个脚印:使用Milestone进行长期视觉和语言导航

One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestones

The Ohio State University(俄亥俄州立大学);U.S. Army Research Laboratory(美国陆军研究实验室)

Chan Hee Song, Jihyung Kil, Tai-Yu Pan, Brian M. Sadler, Wei-Lun Chao, Yu Su

摘要:

我们研究的问题,发展自主智能体,可以遵循人类的指示,推断和执行一系列的行动,以完成底层任务。近年来取得了重大进展,特别是在短期任务方面。然而,当涉及到具有扩展动作序列的长时间任务时,智能体很容易忽略一些指令,或者卡在长指令的中间,最终导致任务失败。为了解决这一挑战,我们提出了一个模型不可知的基于里程碑的任务跟踪器(M-TRACK),以指导代理和监控其进度。具体来说,我们提出了一个里程碑建设者,标记的指示与导航和互动的里程碑,代理需要一步一步地完成,和一个里程碑检查,系统地检查代理的进展,在其当前的里程碑,并确定何时进行到下一个。在具有挑战性的ALFRED数据集上,我们的M-TRACK在两个竞争性基础模型上,在看不见的成功率方面分别显著提高了33%和52%。

英文摘要:

We study the problem of developing autonomous agents that can follow human instructions to infer and perform a sequence of actions to complete the underlying task. Significant progress has been made in recent years, especially for tasks with short horizons. However, when it comes to long-horizon tasks with extended sequences of actions, an agent can easily ignore some instructions or get stuck in the middle of the long instructions and eventually fail the task. To address this challenge, we propose a model-agnostic milestone-based task tracker (M-TRACK) to guide the agent and monitor its progress. Specifically, we propose a milestone builder that tags the instructions with navigation and interaction milestones which the agent needs to complete step by step, and a milestone checker that systemically checks the agent's progress in its current milestone and determines when to proceed to the next. On the challenging ALFRED dataset, our M-TRACK leads to a notable 33% and 52% relative improvement in unseen success rate over two competitive base models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑