arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.13181cs.ROcs.LG
收藏

DINOBot:通过视觉基础模型的检索与对齐实现机器人操控

DINOBot: Robot Manipulation via Retrieval and Alignment with Vision Foundation Models

Imperial College London(伦敦帝国学院)

Norman Di Palo, Edward Johns

AI总结:

DINOBot 提出一种基于 DINO 视觉 Transformer 特征的模仿学习框架,通过检索相似物体并对齐末端执行器,实现高效且泛化的机器人操控。

摘要:

我们提出 DINOBot,一种用于机器人操控的新型模仿学习框架,它利用从使用 DINO 训练的视觉 Transformer 中提取的特征的图像级和像素级能力。在与新物体交互时,DINOBot 首先使用这些特征检索在人类演示期间经历过的最视觉相似的物体,然后使用该物体将其末端执行器与新物体对齐,以实现有效交互。通过对日常任务的一系列真实世界实验,我们表明利用视觉基础模型的图像级和像素级特性能够实现前所未有的学习效率和泛化能力。视频和代码可在 https://www.robot-learning.uk/dinobot 获取。

英文摘要:

We propose DINOBot, a novel imitation learning framework for robot manipulation, which leverages the image-level and pixel-level capabilities of features extracted from Vision Transformers trained with DINO. When interacting with a novel object, DINOBot first uses these features to retrieve the most visually similar object experienced during human demonstrations, and then uses this object to align its end-effector with the novel object to enable effective interaction. Through a series of real-world experiments on everyday tasks, we show that exploiting both the image-level and pixel-level properties of vision foundation models enables unprecedented learning efficiency and generalisation. Videos and code are available at https://www.robot-learning.uk/dinobot.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑