Feasibility Retargeting for Multi-contact Teleoperation and Physical Interaction
Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Université de Lorraine（洛林大学）；University of Edinburgh（爱丁堡大学）；University College London（伦敦大学学院）；University of Sussex（萨塞克斯大学）
This short paper outlines two recent works on multi-contact teleoperation and the development of the SEIKO (Sequential Equilibrium Inverse Kinematic Optimization) framework. SEIKO adapts commands from the operator in real-time and ensures that the reference configuration sent to the underlying controller is feasible. Additionally, an admittance scheme is used to implement physical interaction, which is then combined with the operator's command and retargeted. SEIKO has been applied in simulations on various robots, including humanoid and quadruped robots designed for loco-manipulation. Furthermore, SEIKO has been tested on real hardware for bimanual heavy object carrying tasks.
DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control
School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen（人工智能学院，香港中文大学（深圳））；School of Computing, University of Portsmouth（计算学院，朴茨茅斯大学）
Istituto Italiano di Tecnologia（意大利理工学院）；University of Genova（热那亚大学）；The University of Manchester（曼彻斯特大学）
Humanoid robot teleoperation allows humans to integrate their cognitive capabilities with the apparatus to perform tasks that need high strength, manoeuvrability and dexterity. This paper presents a framework for teleoperation of humanoid robots using a novel approach for motion retargeting through inverse kinematics over the robot model. The proposed method enhances scalability for retargeting, i.e., it allows teleoperating different robots by different human users with minimal changes to the proposed system. Our framework enables an intuitive and natural interaction between the human operato…
Unsupervised Neural Motion Retargeting for Humanoid Teleoperation
Graduate School of Informatics, Kyoto University（京都大学 informatics 研究生院）；Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST)（独立行政法人 产业技术综合研究所 人工智能研究中心）；Advanced Telecommunications Research Institute International (ATR)（国际电气通信基础技术研究所）；Graduate School of Science Technology, Nara Institute of Science and Technology（奈良科学技术研究所 研究生院）
Zhejiang University（浙江大学）；ABB Corporate Research Center（ABB企业研究中心）
We aim to develop an efficient programming method for equipping service robots with the skill of performing sign language motions. This paper addresses the problem of transferring complex dual-arm sign language motions characterized by the coordination among arms and hands from human to robot, which is seldom considered in previous studies of motion retargeting techniques. In this paper, we propose a novel motion retargeting method that leverages graph optimization and Dynamic Movement Primitives (DMPs) for this problem. We employ DMPs in a leader-follower manner to parameterize the original t…
DexTwist: Dexterous Hand Retargeting for Twist Motion via Mixed Reality-based Teleoperation
Autonomous Systems Lab, Institute of Computer Technology, TU Wien（自主系统实验室，计算机技术学院，维也纳技术大学）；Institute of Robotics and Mechatronics (DLR), German Aerospace Center（机器人与机电系统研究所（DLR），德国航空航天中心）
Digital Twins Below the Surface: Enhancing Underwater Teleoperation
School of Engineering and Physical Sciences, Heriot-Watt University（赫瑞瓦特大学工程与物理科学学院）；School of Mathematics and Computer Sciences, Heriot-Watt University（赫瑞瓦特大学数学与计算机科学学院）
This paper discusses the feasibility of using configuration space (C-space) as a means of visualization and control in operator-guided real-time motion of a robot arm manipulator. The motivation is to improve performance of the human operator in tasks involving the manipulator motion in an environment with obstacles. Unlike some other motion planning tasks, operators are known to make expensive mistakes in such tasks, even in a simpler two-dimensional case. They have difficulty learning better procedures and their performance improves very little with practice. Using an example of a two-dimens…
Analyzing Key Objectives in Human-to-Robot Retargeting for Dexterous Manipulation
Department of Automation, Beijing National Research Center for Information Science and Technology, Tsinghua University, China（自动化系，信息科学与技术国家工程研究中心，清华大学）
Georgia Institute of Technology（佐治亚理工学院）；Twentieth Century Fox（二十世纪福克斯公司）；Starfleet Academy（星际舰队学院）；Tyrell Inc.（泰伦公司）
This paper presents a multi-contact motion adaptation framework that enables teleoperation of high degree-of-freedom (DoF) robots, such as quadrupeds and humanoids, for loco-manipulation tasks in multi-contact settings. Our proposed algorithms optimize whole-body configurations and formulate the retargeting of multi-contact motions as sequential quadratic programming, which is robust and stable near the edges of feasibility constraints. Our framework allows real-time operation of the robot and reduces cognitive load for the operator because infeasible commands are automatically adapted into ph…
Bootstrapping Robotic Skill Learning With Intuitive Teleoperation: Initial Feasibility Study
The Chinese University of Hong Kong（香港中文大学）；Multi-Scale Medical Robotics Center（多尺度医疗机器人中心）；Technical University of Munich（慕尼黑工业大学）；Cornerstone Robotics Limited（基石机器人有限公司）
TypeTele: Releasing Dexterity in Teleoperation by Dexterous Manipulation Types
School of Computer Science and Engineering, Sun Yat-sen University, China（中山大学计算机科学与工程学院）；Stanford University, USA（斯坦福大学）；Imperial College London, UK（伦敦帝国理工学院）
Human2Humanoid: Physics-Aware Cross-Morphology Motion Retargeting for Humanoid Robots
Institute of Humanoid Robots, Department of Precision Machinery and Precision Instrumentation, University of Science and Technology of China（人形机器人研究院，精密机械与精密仪器系，中国科学技术大学）
CaFe-TeleVision: A Coarse-to-Fine Teleoperation System with Immersive Situated Visualization for Enhanced Ergonomics
Department of Mechanical and Automation Engineering, T-Stone Robotics Institute, The Chinese University of Hong Kong, Hong Kong SAR（机械与自动化工程系、T-Stone机器人研究所、香港中文大学（深圳））
Spatio-Temporal Motion Retargeting for Quadruped Robots
Department of Artificial Intelligence, Korea University（人工智能系，韩国大学）；Department of Computer Science, ETH Zurich（计算机科学系，苏黎世联邦理工学院）；Department of Mechanical and Aerospace Engineering, UCLA（机械与航空航天工程系，加州大学洛杉矶分校）
Dept. of Mechanical Engineering（机械工程系）；Dept. of Computer Science, Johns Hopkins University（约翰斯·霍普金斯大学计算机科学系）
Teleoperation platforms often require the user to be situated at a fixed location to both visualize and control the movement of the robot and thus do not provide the operator with much mobility. One example is in existing robotic surgery solutions that require the surgeons to be away from the patient, attached to consoles where their heads must be fixed and their arms can only move in a limited space. This creates a barrier between physicians and patients that does not exist in normal surgery. To address this issue, we propose a mobile telesurgery solution where the surgeons are no longer mech…
Generating Humanoid Multi-Contact through Feasibility Visualization
Institute of Human and Machine Cognition (IHMC)（人类与机器认知研究所）；University of West Florida (UWF)（西佛罗里达大学）；Georgia Institute of Technology（佐治亚理工学院）；Figure AI, Inc.（Figure AI公司）
We propose to perform imitation learning for dexterous manipulation with multi-finger robot hand from human demonstrations, and transfer the policy to the real robot hand. We introduce a novel single-camera teleoperation system to collect the 3D demonstrations efficiently with only an iPad and a computer. One key contribution of our system is that we construct a customized robot hand for each user in the physical simulator, which is a manipulator resembling the same kinematics structure and shape of the operator's hand. This provides an intuitive interface and avoid unstable human-robot hand r…
The Cloud-based Advanced Robotics Laboratory (CARL) integrates a whole body controller and web-based teleoperation to enable any device with a web browser to access and control a humanoid robot. By integrating humanoid robots with the cloud, they are accessible from any Internet-connected device. Increased accessibility is important because few people have access to state-of-the-art humanoid robots limiting their rate of development. CARL's implementation is based on modern software libraries, frameworks, and middleware including Node.js, Socket.IO, ZMQ, ROS, Robot Web Tools, and ControlIt! Fe…
KineDex: Learning Tactile-Informed Visuomotor Policies via Kinesthetic Teaching for Dexterous Manipulation
Tongji University（同济大学）；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；University of Hong Kong（香港大学）；Shanghai Qi Zhi Institute（上海启智研究院）；Shanghai AI Lab（上海人工智能实验室）
