arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.11235cs.ROcs.AI
收藏

面向高效机器人学习的空间-语言注意力策略

Spatial-Language Attention Policies for Efficient Robot Learning

Meta;Carnegie Mellon(卡内基梅隆大学);SUNY Binghamton(纽约州立大学宾汉姆顿分校)

Priyam Parashar, Vidhi Jain, Xiaohan Zhang, Jay Vakil, Sam Powers, Yonatan Bisk, Chris Paxton

AI总结:

针对现有语言引导机器人操作局限于桌面场景、移动操作下视角与抓取变化鲁棒性不足且数据效率低的问题,提出使用三维token的空间-语言注意力策略SLAP,在多任务移动操作中实现了性能与数据效率的显著提升。

摘要:

尽管语言引导的操作已取得长足进展,但现有研究仍局限于桌面场景。桌面场景可提供稳定一致的理想拍摄角度,而这一特性在移动操作场景中并不成立。涉及在环境中移动的任务规划必须能适应第一人称视角,以及抓取平面与角度的变化。另一项挑战是,在保证上述适应性的同时,仍能从有限数据中高效学习技能。我们提出空间-语言注意力策略(Spatial-Language Attention Policies,SLAP)作为解决方案。SLAP使用三维token作为输入表示,训练单个多任务、以语言为条件的动作预测策略。我们的方法在真实世界8项任务中,单个模型即可达到80%的成功率;在引入未见过的杂乱场景与未见过的物体配置时,即便每个任务仅使用少量示例,成功率仍达47.5%。这比现有工作提升了30%(在存在未见过的干扰物与配置的场景下,现有工作成功率为20%)。在移动操作场景中,我们的方法较基线提升了4倍。此外,我们展示了SLAP的鲁棒性如何支持通过大语言模型解析开放词汇指令,执行多步移动操作的任务规划。相关视频见网站:https://robotslap.github.io

英文摘要:

Despite great strides in language-guided manipulation, existing work has been constrained to table-top settings. Table-tops allow for perfect and consistent camera angles, properties are that do not hold in mobile manipulation. Task plans that involve moving around the environment must be robust to egocentric views and changes in the plane and angle of grasp. A further challenge is ensuring this is all true while still being able to learn skills efficiently from limited data. We propose Spatial-Language Attention Policies (SLAP) as a solution. SLAP uses three-dimensional tokens as the input representation to train a single multi-task, language-conditioned action prediction policy. Our method shows an 80% success rate in the real world across eight tasks with a single model, and a 47.5% success rate when unseen clutter and unseen object configurations are introduced, even with only a handful of examples per task. This represents an improvement of 30% over prior work (20% given unseen distractors and configurations). We see a 4x improvement over baseline in mobile manipulation setting. In addition, we show how SLAPs robustness allows us to execute Task Plans from open-vocabulary instructions using a large language model for multi-step mobile manipulation. For videos, see the website: https://robotslap.github.io

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑