RDT-1B：一种面向双臂操作的扩散基础模型
RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation
AI总结：
针对双臂操作的多模态动作分布与数据稀缺挑战，提出了基于扩散模型和可扩展Transformer的1.2B参数基础模型RDT，并引入物理可解释统一动作空间，在真实机器人实验中展现出卓越的零样本泛化与少样本学习能力。
摘要：
双臂操作在机器人技术中至关重要，然而由于协调两个机械臂带来的固有复杂性（导致多模态动作分布）以及训练数据的稀缺性，开发基础模型极具挑战性。本文提出了 Robotics Diffusion Transformer (RDT)，一种面向双臂操作的开创性扩散基础模型。RDT 基于扩散模型以有效表示多模态，并对可扩展 Transformer 进行了创新设计，以处理多模态输入的异构性，并捕捉机器人数据的非线性和高频特性。为解决数据稀缺问题，我们进一步引入了 Physically Interpretable Unified Action Space，该空间能统一各种机器人的动作表示，同时保留原始动作的物理意义，从而促进可迁移物理知识的学习。基于这些设计，我们在迄今最大规模的多机器人数据集集合上对 RDT 进行了预训练，并将其扩展至 1.2B 参数，这是用于机器人操作的最大基于扩散的基础模型。最后，我们在一个自建的包含超过 6K+ 轨迹的多任务双臂数据集上对 RDT 进行了微调，以精炼其操作能力。真实机器人实验表明，RDT 显著优于现有方法。它展现出对未见物体和场景的零样本泛化能力，能够理解并遵循语言指令，仅需 1~5 次演示即可学习新技能，并能有效处理复杂的灵巧任务。代码和视频请参阅 https://rdt-robotics.github.io/rdt-robotics/。
英文摘要：
Bimanual manipulation is essential in robotics, yet developing foundation models is extremely challenging due to the inherent complexity of coordinating two robot arms (leading to multi-modal action distributions) and the scarcity of training data. In this paper, we present the Robotics Diffusion Transformer (RDT), a pioneering diffusion foundation model for bimanual manipulation. RDT builds on diffusion models to effectively represent multi-modality, with innovative designs of a scalable Transformer to deal with the heterogeneity of multi-modal inputs and to capture the nonlinearity and high frequency of robotic data. To address data scarcity, we further introduce a Physically Interpretable Unified Action Space, which can unify the action representations of various robots while preserving the physical meanings of original actions, facilitating learning transferrable physical knowledge. With these designs, we managed to pre-train RDT on the largest collection of multi-robot datasets to date and scaled it up to 1.2B parameters, which is the largest diffusion-based foundation model for robotic manipulation. We finally fine-tuned RDT on a self-created multi-task bimanual dataset with over 6K+ episodes to refine its manipulation capabilities. Experiments on real robots demonstrate that RDT significantly outperforms existing methods. It exhibits zero-shot generalization to unseen objects and scenes, understands and follows language instructions, learns new skills with just 1~5 demonstrations, and effectively handles complex, dexterous tasks. We refer to https://rdt-robotics.github.io/rdt-robotics/ for the code and videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
