To represent motions from a mechanical point of view, this paper explores motion embedding using the motion taxonomy. With this taxonomy, manipulations can be described and represented as binary strings called motion codes. Motion codes capture mechanical properties, such as contact type and trajectory, that should be used to define suitable distance metrics between motions or loss functions for deep learning and reinforcement learning. Motion codes can also be used to consolidate aliases or cluster motion types that share similar properties. Using existing data sets as a reference, we discuss how motion codes can be created and assigned to actions that are commonly seen in activities of daily living based on intuition as well as real data. Motion codes are compared to vectors from pre-trained Word2Vec models, and we show that motion codes maintain distances that closely match the reality of manipulation.
DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulation
DexRobot Co. Ltd.(德克斯机器人有限公司);University of Michigan(密歇根大学);Shanghai Jiao Tong University(上海交通大学);Chongqing University(重庆大学);East China University of Science and Technology(东华大学)
Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges
Ecole Centrale de Lyon, CNRS, Universite Claude Bernard Lyon 1, INSA Lyon, Université Lumière Lyon 2, LIRIS, UMR5205(Lyon中央理工学院、CNRS、Claude Bernard Lyon 1大学、INSA Lyon、Lyon大学、LIRIS、UMR5205);Beihang University(北航大学);Dalian University of Technology(大连理工大学);Institut Universitaire de France (IUF)(法国大学研究院(IUF))
Towards a Unified Understanding of Robot Manipulation: A Comprehensive Survey
Xi’an Jiaotong University(西安交通大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Chinese Academy of Sciences(中国科学院);Westlake University(西湖大学);Zhejiang University(浙江大学);University of Sydney(悉尼大学);BAAI(北京智源人工智能研究院);Peking University(北京大学)
PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Korea Advanced Institute of Science and Technology(韩国科学技术院);ETH Zurich(苏黎世联邦理工学院)
Instruction-Augmented Long-Horizon Planning: Embedding Grounding Mechanisms in Embodied Mobile Manipulation
The Hong Kong Polytechnic University(香港理工大学);Ningbo Institute of Digital Twin, Eastern Institute of Technology(东方理工大学宁波数字孪生研究院);Great Bay University(大湾区大学);Mohamed Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)