arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.15980cs.ROcs.AI
收藏

基于下一词元预测的上下文模仿学习

In-Context Imitation Learning via Next-Token Prediction

University of California, Berkeley(加州大学伯克利分校);Autodesk Research(欧特克研究中心)

Letian Fu, Huang Huang, Gaurav Datta, Lawrence Yunliang Chen, William Chung-Ho Panitch, Fangchen Liu, Hui Li, Ken Goldberg

AI总结:

本文提出上下文机器人Transformer(ICRT),通过下一词元预测对感觉运动轨迹进行自回归建模,无需语言数据或奖励函数即可在真实机器人上实现免训练的上下文模仿学习,并在多任务泛化中显著优于现有方法。

摘要:

我们探索如何增强下一词元预测模型,使其在真实机器人上执行上下文模仿学习,其中机器人在输入阶段通过解读所提供的上下文信息来执行新任务,而无需更新其底层策略参数。我们提出上下文机器人Transformer(ICRT),这是一种因果Transformer,在感觉运动轨迹上执行自回归预测,不依赖任何语言数据或奖励函数。该公式使得在测试时能够灵活且免训练地执行新任务,通过向模型提示新任务的感觉运动轨迹来实现,这些轨迹由图像观测、动作和状态三元组组成,并通过人类遥操作收集。使用Franka Emika机器人的实验表明,ICRT能够适应由提示指定的新任务,即使在环境配置与提示和训练数据均不同的情况下也是如此。在多任务环境设置中,ICRT在泛化到未见任务方面显著优于当前机器人领域最先进的下一词元预测模型。代码、检查点和数据可在 https://icrt.dev/ 获取。

英文摘要:

We explore how to enhance next-token prediction models to perform in-context imitation learning on a real robot, where the robot executes new tasks by interpreting contextual information provided during the input phase, without updating its underlying policy parameters. We propose In-Context Robot Transformer (ICRT), a causal transformer that performs autoregressive prediction on sensorimotor trajectories without relying on any linguistic data or reward function. This formulation enables flexible and training-free execution of new tasks at test time, achieved by prompting the model with sensorimotor trajectories of the new task composing of image observations, actions and states tuples, collected through human teleoperation. Experiments with a Franka Emika robot demonstrate that the ICRT can adapt to new tasks specified by prompts, even in environment configurations that differ from both the prompt and the training data. In a multitask environment setup, ICRT significantly outperforms current state-of-the-art next-token prediction models in robotics on generalizing to unseen tasks. Code, checkpoints and data are available on https://icrt.dev/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑