RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot Hands
University of Oulu(奥卢大学);Aalto University(阿尔托大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);University of Southern California(南加利福尼亚大学)
Yi Zhao, Le Chen, Jan Schneider, Quankai Gao, Juho Kannala, Bernhard Schölkopf, Joni Pajarinen, Dieter Büchler
赋予机器人手人类水平的灵巧性一直是一个长期的研究目标。双臂机器人弹钢琴构成了一项任务,它结合了动态任务的挑战(如生成快速且精确的运动)与较慢但接触丰富的操作问题。尽管基于强化学习的方法在单任务性能上展现出了有前景的结果,但这些方法在多曲目设定下表现吃力。我们的工作旨在弥合这一差距,从而使得大规模的机器人弹钢琴模仿学习方法成为可能。为此,我们引入了 Robot Piano 1 Million (RP1M) 数据集,包含超过一百万条轨迹的双臂机器人弹钢琴运动数据。我们将手指放置形式化为最优传输问题,从而能够对大量未标注曲目进行自动标注。对现有模仿学习方法的基准测试表明,这些方法通过利用 RP1M 达到了当前最先进的机器人弹钢琴性能。
英文摘要:
It has been a long-standing research goal to endow robot hands with human-level dexterity. Bi-manual robot piano playing constitutes a task that combines challenges from dynamic tasks, such as generating fast while precise motions, with slower but contact-rich manipulation problems. Although reinforcement learning based approaches have shown promising results in single-task performance, these methods struggle in a multi-song setting. Our work aims to close this gap and, thereby, enable imitation learning approaches for robot piano playing at scale. To this end, we introduce the Robot Piano 1 Million (RP1M) dataset, containing bi-manual robot piano playing motion data of more than one million trajectories. We formulate finger placements as an optimal transport problem, thus, enabling automatic annotation of vast amounts of unlabeled songs. Benchmarking existing imitation learning approaches shows that such approaches reach state-of-the-art robot piano playing performance by leveraging RP1M.
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Aalto University(阿alto大学);University of Southern California(南加州大学);Imperial College London(伦敦帝国理工学院);University of Oulu(奥卢大学);University of Alberta(阿尔伯塔大学);Alberta Machine Intelligence Institute(阿尔伯塔人工智能研究所)
DigiArm: An Anthropomorphic 3D-Printed Prosthetic Hand with Enhanced Dexterity for Typing Tasks
Department of Computer Science, Technion, Haifa, Israel(计算机科学系,技术学院,海法,以色列);Department of Mechanical Engineering, Technion, Haifa, Israel(机械工程系,技术学院,海法,以色列)
SynManDex: Synthesizing Human-like Dexterous Grasps from Synthetic Human Pre-Grasps
Shanghai AI Lab(上海人工智能实验室);Shanghai Jiaotong University(上海交通大学);Shenzhen University(深圳大学);Fudan University(复旦大学);University of Hong Kong(香港大学);ZTE Corporation(中兴通讯股份有限公司)
A Haptic Robot Finger Designed for Guqin Instrument Playing
The Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳市人工智能与机器人研究院);The Chinese University of Hong Kong - Shenzhen(香港中文大学(深圳));School of Physics and Optoelectronic Engineering, Shenzhen University(深圳大学物理与光电工程学院);Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)