Efficient and intuitive Human-Robot interfaces are crucial for expanding the user base of operators and enabling new applications in critical areas such as precision agriculture, automated construction, rehabilitation, and environmental monitoring. In this paper, we investigate the design of human-robot interfaces for the teleoperation of dynamical systems. The proposed framework seeks to find an optimal interface that complies with key concepts such as user comfort, efficiency, continuity, and consistency. As a proof-of-concept, we introduce an innovative approach to teleoperating underwater vehicles, allowing the translation between human body movements into vehicle control commands. This method eliminates the need for divers to work in harsh underwater environments while taking into account comfort and communication constraints. We conducted a study with human subjects using a head-mounted display attached to a smartphone to control a simulated ROV. Also, numerical experiments have demonstrated that the optimal translation is often the most intuitive and natural one, aligning with users' expectations.
Digital Twins Below the Surface: Enhancing Underwater Teleoperation
School of Engineering and Physical Sciences, Heriot-Watt University（赫瑞瓦特大学工程与物理科学学院）；School of Mathematics and Computer Sciences, Heriot-Watt University（赫瑞瓦特大学数学与计算机科学学院）
IMMERTWIN: A Mixed Reality Framework for Enhanced Robotic Arm Teleoperation
School of Computing Science, University of Glasgow（格拉斯哥大学计算科学学院）；ICPS Research Center, National Institute of Advanced Industrial Science and Technology (AIST)（日本国立先进工业科学技术研究院ICPS研究中心）
Confidence-based Intent Prediction for Teleoperation in Bimanual Robotic Suturing
Department of Mechanical Engineering, Imperial College London（机械工程系，帝国理工学院伦敦分校）；Department of Surgery & Cancer, Imperial College London（外科与癌症系，帝国理工学院伦敦分校）；Department of Engineering and Design, University of Sussex（工程与设计系，Sussex大学）；Department of Bioengineering, Imperial College London（生物工程系，帝国理工学院伦敦分校）
Telepresence and teleoperation robotics have attracted a great amount of attention in the last 10 years. With the Artificial Intelligence (AI) revolution already being started, we can see a wide range of robotic applications being realized. Intelligent robotic systems are being deployed both in industrial and domestic environments. Telepresence is the idea of being present in a remote location virtually or via robotic avatars. Similarly, the idea of operating a robot from a remote location for various tasks is called teleoperation. These technologies find significant application in health care…
This paper presents a robotics vision-based heuristic reasoning system for underwater target tracking and navigation. This system is introduced to improve the level of automation of underwater Remote Operated Vehicles (ROVs) operations. A prototype which combines computer vision with an underwater robotics system is successfully designed and developed to perform target tracking and intelligent navigation. ...
We propose a teleoperation system that uses a single RGB-D camera as the human motion capture device. Our system can perform general manipulation tasks such as cloth folding, hammering and 3mm clearance peg in hole. We propose the use of non-Cartesian oblique coordinate frame, dynamic motion scaling and reposition of operator frames to increase the flexibility of our teleoperation system. We hypothesize that lowering the barrier of entry to teleoperation will allow for wider deployment of supervised autonomy system, which will in turn generates realistic datasets that unlock the potential of m…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Teleoperation (i.e., controlling a robot with human motion) proves promising in enabling a humanoid robot to move as dynamically as a human. But how to map human motion to a humanoid robot matters because a human and a humanoid robot rarely have identical topologies and dimensions. This work presents an experimental study that utilizes reaction tests to compare the proposed joint space mapping and the proposed task space mapping for dynamic teleoperation of an anthropomorphic robotic arm that possesses human-level dynamic motion capabilities. The experimental results suggest that the robot ach…
Bootstrapping Robotic Skill Learning With Intuitive Teleoperation: Initial Feasibility Study
The Chinese University of Hong Kong（香港中文大学）；Multi-Scale Medical Robotics Center（多尺度医疗机器人中心）；Technical University of Munich（慕尼黑工业大学）；Cornerstone Robotics Limited（基石机器人有限公司）
UniBiDex: A Unified Teleoperation Framework for Robotic Bimanual Dexterous Manipulation
Department of Computer Science, The University of Hong Kong（香港大学计算机科学系）；Dongguan Key Laboratory of Intelligent Equipment and Smart Industry, School of Advanced Engineering, The Great Bay University（东莞智能装备与智能制造重点实验室，高级工程学院，大湾区大学）；Department of Mechanical Engineering, The Hong Kong Polytechnic University（香港理工大学机械工程系）；Guangdong Academy of Sciences（广东省科学院）
Model Predictive Control (MPC) schemes have proven their efficiency in controlling high degree-of-freedom (DoF) complex robotic systems. However, they come at a high computational cost and an update rate of about tens of hertz. This relatively slow update rate hinders the possibility of stable haptic teleoperation of such systems since the slow feedback loops can cause instabilities and loss of transparency to the operator. This work presents a novel framework for transparent teleoperation of MPC-controlled complex robotic systems. In particular, we employ a feedback MPC approach and exploit i…
Predictive Relative-Velocity Steering for Safe Robotic Manipulator Teleoperation in Dynamic Environments
Zhejiang University（浙江大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Tsinghua University（清华大学）；Institute for Embodied Intelligence and Robotics, Tsinghua University（清华大学具身智能与机器人研究所）；TetraBOT
Teleoperation offers the possibility of imparting robotic systems with sophisticated reasoning skills, intuition, and creativity to perform tasks. However, current teleoperation solutions for high degree-of-actuation (DoA), multi-fingered robots are generally cost-prohibitive, while low-cost offerings usually provide reduced degrees of control. Herein, a low-cost, vision based teleoperation system, DexPilot, was developed that allows for complete control over the full 23 DoA robotic system by merely observing the bare human hand. DexPilot enables operators to carry out a variety of complex man…
Universität Hamburg（汉堡大学）；Tsinghua University（清华大学）
In this paper, we present TeachNet, a novel neural network architecture for intuitive and markerless vision-based teleoperation of dexterous robotic hands. Robot joint angles are directly generated from depth images of the human hand that produce visually similar robot hand poses in an end-to-end fashion. The special structure of TeachNet, combined with a consistency loss function, handles the differences in appearance and anatomy between human and robotic hands. A synchronized human-robot training set is generated from an existing dataset of labeled depth images of the human hand and simulate…
TeleHairing: A Teleoperation Baseline for Robotic Haircutting
University of Oulu（奥卢大学）；Shaoyang University（邵阳学院）；Jishou University（吉首大学）；University of Exeter（埃克塞特大学）；University of Naples Federico II（那不勒斯腓特烈二世大学）；VTT Technical Research Centre of Finland（芬兰国家技术研究中心）
Italian National Agency for New Technologies, Energy and Sustainable Economic Development（意大利新技术、能源与可持续经济发展署）；Archeomatica（考古数学机构）
Cultural Heritage can largely profit from the set of technologies that have recently been developed in Submarine Robotics. In this paper we focus on how underwater robotics and related technologies can be used to enhance economical fruition, control, protection and social impact of the cultural heritage. Robots allow on-line experience, in remote locations, realizing the remote museum concept as extension and enhancement of the virtual museum. These solutions push the cultural tourism beyond actual limits of the sites like the number of simultaneous visitors, the travelling costs, , the diffic…
Robotic teleoperation brings great potential for advances within the field of surgery. The ability of a surgeon to reach patient remotely opens exciting opportunities. Early experience with telerobotic surgery has been interesting, but the clinical feasibility remains out of reach, largely due to the deleterious effects of communication delays. Teleoperation tasks are significantly impacted by unavoidable signal latency, which directly results in slower operations, less precision in movements, and increased human errors. Introducing significant changes to the surgical workflow, for example by …
Dept. of Mechanical Engineering（机械工程系）；Dept. of Computer Science, Johns Hopkins University（约翰斯·霍普金斯大学计算机科学系）
Teleoperation platforms often require the user to be situated at a fixed location to both visualize and control the movement of the robot and thus do not provide the operator with much mobility. One example is in existing robotic surgery solutions that require the surgeons to be away from the patient, attached to consoles where their heads must be fixed and their arms can only move in a limited space. This creates a barrier between physicians and patients that does not exist in normal surgery. To address this issue, we propose a mobile telesurgery solution where the surgeons are no longer mech…
A Vision-Based Shared-Control Teleoperation Scheme for Controlling the Robotic Arm of a Four-Legged Robot
Department of Mechanical Engineering, University of São Paulo（机械工程系，圣保罗大学）；Instituto Israelita de Ensino e Pesquisa Albert Einstein, Hospital Israelita Albert Einstein（艾利哈德研究所，以色列医院）
Raspi$^2$USBL: An open-source Raspberry Pi-Based Passive Inverted Ultra-Short Baseline Positioning System for Underwater Robotics
State Key Laboratory of Ocean Sensing, Ocean College of Zhejiang University（浙江省海洋传感重点实验室，浙江大学海洋学院）；School of Oceanography, Shanghai Jiao Tong University（上海交通大学海洋学院）
Graduate School of Engineering Science, Osaka University（大阪大学工学研究科）；City University of Hongkong（香港城市大学）
We present bilateral teleoperation system for task learning and robot motion generation. Our system includes a bilateral teleoperation platform and a deep learning software. The deep learning software refers to human demonstration using the bilateral teleoperation platform to collect visual images and robotic encoder values. It leverages the datasets of images and robotic encoder information to learn about the inter-modal correspondence between visual images and robot motion. In detail, the deep learning software uses a combination of Deep Convolutional Auto-Encoders (DCAE) over image regions,…
