通过视觉语言模型进行指令增强的机器人技能获取
Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models
AI总结：
本文提出DIAL方法，利用预训练视觉语言模型CLIP的语义理解能力，为大规模未标注机器人演示数据自动生成语言标签，使模仿学习策略能以更低成本获取新能力并泛化至未见指令。
摘要：
近年来，在遵循自然语言指令的机器人操作策略学习方面取得了很大进展。此类方法通常从机器人-语言数据语料库中学习，这些数据要么是针对特定任务收集的，要么是事后由人类用丰富的语言描述进行昂贵的重新标注。最近，像CLIP或ViLD这样的大规模预训练视觉语言模型（VLMs）已被应用于机器人领域，用于学习表示和场景描述器。这些预训练模型能否作为机器人数据的自动标注器，有效地将互联网规模的知识导入现有数据集，使其即使对于未在其真实标注中反映的任务也能发挥作用？为了实现这一点，我们引入了用于语言条件控制的数据驱动指令增强（DIAL）：我们利用CLIP的语义理解能力，使用半监督语言标签将知识传播到大量未标记的演示数据上，然后在增强后的数据集上训练语言条件策略。与昂贵的人类标签相比，该方法能够以更低的成本获取有用的语言描述，从而实现对大规模数据集更高效的标签覆盖。我们将DIAL应用于一个具有挑战性的真实世界机器人操作领域，其中80,000个演示中有96.5%不包含众包语言标注。DIAL使模仿学习策略能够获取新能力，并泛化到原始数据集中未见的60个新指令。
英文摘要：
In recent years, much progress has been made in learning robotic manipulation policies that follow natural language instructions. Such methods typically learn from corpora of robot-language data that was either collected with specific tasks in mind or expensively re-labelled by humans with rich language descriptions in hindsight. Recently, large-scale pretrained vision-language models (VLMs) like CLIP or ViLD have been applied to robotics for learning representations and scene descriptors. Can these pretrained models serve as automatic labelers for robot data, effectively importing Internet-scale knowledge into existing datasets to make them useful even for tasks that are not reflected in their ground truth annotations? To accomplish this, we introduce Data-driven Instruction Augmentation for Language-conditioned control (DIAL): we utilize semi-supervised language labels leveraging the semantic understanding of CLIP to propagate knowledge onto large datasets of unlabelled demonstration data and then train language-conditioned policies on the augmented datasets. This method enables cheaper acquisition of useful language descriptions compared to expensive human labels, allowing for more efficient label coverage of large-scale datasets. We apply DIAL to a challenging real-world robotic manipulation domain where 96.5% of the 80,000 demonstrations do not contain crowd-sourced language annotations. DIAL enables imitation learning policies to acquire new capabilities and generalize to 60 novel instructions unseen in the original dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
