Hand motion capture data is now relatively easy to obtain, even for complicated grasps; however this data is of limited use without the ability to retarget it onto the hands of a specific character or robot. The target hand may differ dramatically in geometry, number of degrees of freedom (DOFs), or number of fingers. We present a simple, but effective framework capable of kinematically retargeting multiple human hand-object manipulations from a publicly available dataset to a wide assortment of kinematically and morphologically diverse target hands through the exploitation of contact areas. We do so by formulating the retarget operation as a non-isometric shape matching problem and use a combination of both surface contact and marker data to progressively estimate, refine, and fit the final target hand trajectory using inverse kinematics (IK). Foundational to our framework is the introduction of a novel shape matching process, which we show enables predictable and robust transfer of contact data over full manipulations while providing an intuitive means for artists to specify correspondences with relatively few inputs. We validate our framework through thirty demonstrations across five different hand shapes and six motions of different objects. We additionally compare our method against existing hand retargeting approaches. Finally, we demonstrate our method enabling novel capabilities such as object substitution and the ability to visualize the impact of design choices over full trajectories.
Feasibility Retargeting for Multi-contact Teleoperation and Physical Interaction
Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);Université de Lorraine(洛林大学);University of Edinburgh(爱丁堡大学);University College London(伦敦大学学院);University of Sussex(萨塞克斯大学)
OTRetarget: Joint Robot and Object Motion Retargeting via Optimal Transport
Inria, Département d’Informatique de l’École Normale Supérieure, PSL Research University(法国国家信息与自动化研究所,巴黎高等师范学校计算机科学系,巴黎文理研究大学);Stanford University(斯坦福大学)
Physics-Driven Data Generation for Contact-Rich Manipulation via Trajectory Optimization
Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology(计算机科学与人工智能实验室(CSAIL),麻省理工学院);Robotics and AI Institute(机器人与人工智能研究院)
SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergy
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院);Department of Automation, Tsinghua University(清华大学自动化系);College of Artificial Intelligence, Nanjing Forestry University(南京林业大学人工智能学院)