Attaching a robotic manipulator to a flying base allows for significant improvements in the reachability and versatility of manipulation tasks. In order to explore such systems while taking advantage of human capabilities in terms of perception and cognition, bilateral teleoperation arises as a reasonable solution. However, since most telemanipulation tasks require visual feedback in addition to the haptic one, real-time (task-dependent) positioning of a video camera, which is usually attached to the flying base, becomes an additional objective to be fulfilled. Since the flying base is part of the kinematic structure of the robot, if proper care is not taken, moving the video camera could undesirably disturb the end-effector motion. For that reason, the necessity of controlling the base position in the null space of the manipulation task arises. In order to provide the operator with meaningful information about the limits of the allowed motions in the null space, this paper presents a novel haptic concept called Null-Space Wall. In addition, a framework to allow stable bilateral teleoperation of both tasks is presented. Numerical simulation data confirm that the proposed framework is able to keep the system passive while allowing the operator to perform time-delayed telemanipulation and command the base to a task-dependent optimal pose.
Wuhan University（武汉大学）；Idiap Research Institute（Idiap研究所）
Mapping operator motions to a robot is a key problem in teleoperation. Due to differences between workspaces, such as object locations, it is particularly challenging to derive smooth motion mappings that fulfill different goals (e.g. picking objects with different poses on the two sides or passing through key points). Indeed, most state-of-the-art methods rely on mode switches, leading to a discontinuous, low-transparency experience. In this paper, we propose a unified formulation for position, orientation and velocity mappings based on the poses of objects of interest in the operator and rob…
Composite Adaptive Control for Bilateral Teleoperation Systems without Persistency of Excitation
School of Automation and Electrical Engineering, University of Science and Technology Beijing（自动化与电气工程学院，北京科技大学）；Department of Automatic Control, Lund University, P.O. Box 118, 22100 Lund, Sweden（自动控制系，卢德大学）
Design and Experimental Validation of Sensorless 4-Channel Bilateral Teleoperation for Low-Cost Manipulators
Degree Programs in Intelligent and Mechanical Interaction Systems, University of Tsukuba（智能与机械交互系统专业，东京大学）；Faculty of Engineering, Information and Systems, University of Tsukuba（工程、信息与系统学部，东京大学）；Department of Electrical Engineering, Electronics, and Applied Physics, Saitama University（电子工程、电子学与应用物理系，埼玉大学）
Bilateral teleoperation offers an intriguing solution towards shared autonomy with aerial vehicles in contact-based inspection and manipulation tasks. Omnidirectional aerial robots allow for full pose operations, making them particularly attractive in such tasks. Naturally, the question arises whether standard bilateral teleoperation methodologies are suitable for use with these vehicles. In this work, a fully decoupled 6DoF bilateral teleoperation framework for aerial physical interaction is designed and tested for the first time. The method is based on the well established rate control, rece…
Graduate School of Engineering Science, Osaka University（大阪大学工学研究科）；City University of Hongkong（香港城市大学）
We present bilateral teleoperation system for task learning and robot motion generation. Our system includes a bilateral teleoperation platform and a deep learning software. The deep learning software refers to human demonstration using the bilateral teleoperation platform to collect visual images and robotic encoder values. It leverages the datasets of images and robotic encoder information to learn about the inter-modal correspondence between visual images and robot motion. In detail, the deep learning software uses a combination of Deep Convolutional Auto-Encoders (DCAE) over image regions,…
Robotics System Lab (RSL), ETH Zurich（苏黎世联邦理工学院机器人系统实验室）；Dynamic Legged Systems Lab, Istituto Italiano di Tecnologia (IIT)（意大利理工学院动态腿足系统实验室）；Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente（特文特大学电气工程、数学与计算机科学学院）
When dealing with the haptic teleoperation of multi-limbed mobile manipulators, the problem of mitigating the destabilizing effects arising from the communication link between the haptic device and the remote robot has not been properly addressed. In this work, we propose a passive control architecture to haptically teleoperate a legged mobile manipulator, while remaining stable in the presence of time delays and frequency mismatches in the master and slave controllers. At the master side, a discrete-time energy modulation of the control input is proposed. At the slave side, passivity constrai…
Input-gated Bilateral Teleoperation: An Easy-to-implement Force Feedback Teleoperation Method for Low-cost Hardware
Research & Development Group, Hitachi, Ltd.（日立株式会社研发部）；Department of Intermedia Art and Science School of Fundamental Science and Engineering, Waseda University（早稻田大学基础科学与工程学部多媒体艺术与科学系）
This paper develops a novel force observer for bilateral teleoperation systems. Type-2 fuzzy logic is used to describe the overall dynamic system, and Moving Horizon Estimation (MHE) is employed to assess clean states as well as the values of dynamic uncertainties, and simultaneously filter out the measurement noises, which guarantee the high degree of accuracy for the observed forces. Compared with the existing methods, the proposed force observer can run without knowing exact mathematical dynamic functions and is robust to different kinds of noises. A force-reflection four-channel teleoperat…
Tampere University（坦佩雷大学）；Canadian Space Agency（加拿大空间局）
This study designs a high-precision bilateral teleoperation control for a dissimilar master-slave system. The proposed nonlinear control design takes advantage of a novel subsystem-dynamics-based control method that allows designing of individual (decentralized) model-based controllers for the manipulators locally at the subsystem level. Very importantly, a dynamic model of the human operator is incorporated into the control of the master manipulator. The individual controllers for the dissimilar master and slave manipulators are connected in a specific communication channel for the bilateral …
This paper explores the possibility of improving bilateral robot manipulation task performance through optimizing the robot morphology and configuration of the system through motion. To optimize the design for different scenarios, we select a set of tasks that represent the variability in small scale manipulation (e.g. pick and place, tasks involving positioning and orientation) and track the motion to obtain a reproducible trajectory. Kinematic data is captured through an electromagnetic (EM) tracker system while a human subject performs the tasks. Then, the data is pre-processed and used to …
LETS: Low-Overhead External-Torque Sensing for Force-Feedback Bilateral Teleoperation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Beijing Amigos Robot Technology Co., Ltd.（北京阿米格斯机器人科技有限公司）
Mobile manipulators in the home can provide increased autonomy to individuals with severe motor impairments, who often cannot complete activities of daily living (ADLs) without the help of a caregiver. Teleoperation of an assistive mobile manipulator could enable an individual with motor impairments to independently perform self-care and household tasks, yet limited motor function can impede one's ability to interface with a robot. In this work, we present a unique inertial-based wearable assistive interface, embedded in a familiar head-worn garment, for individuals with severe motor impairmen…
This paper discusses the feasibility of using configuration space (C-space) as a means of visualization and control in operator-guided real-time motion of a robot arm manipulator. The motivation is to improve performance of the human operator in tasks involving the manipulator motion in an environment with obstacles. Unlike some other motion planning tasks, operators are known to make expensive mistakes in such tasks, even in a simpler two-dimensional case. They have difficulty learning better procedures and their performance improves very little with practice. Using an example of a two-dimens…
A study on a Real-Time VR-Based Teleoperation Framework for Manipulator in Dynamic Environment
Department of Robotics, Hanyang University（韩世大学机器人系）；Department of Smart Construction Engineering, Hanyang University（韩世大学智能建造工程系）；Department of Interdisciplinary Robot Engineering Systems, Hanyang University（韩世大学跨学科机器人工程系统系）；School of Smart Convergence Engineering, Hanyang University, Ansan（韩世大学智能融合工程学院，安山）
Empowering Robot Teleoperation: Exploring the Synergies Between Devices and Manipulator Controllers in a Comparative Study
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, Guangdong, China（香港中文大学（深圳）科学与工程学院）；Department of Mechanical and Aerospace Engineering, Hong Kong University of Science and Technology（香港科学与技术大学机械与航空航天工程系）；Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS), Shenzhen, Guangdong, China（深圳人工智能与机器人社会研究院（AIRS））
Predictive Relative-Velocity Steering for Safe Robotic Manipulator Teleoperation in Dynamic Environments
Zhejiang University（浙江大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Tsinghua University（清华大学）；Institute for Embodied Intelligence and Robotics, Tsinghua University（清华大学具身智能与机器人研究所）；TetraBOT
Haptic bilateral teleoperation system for free-hand dental procedures
Department of Excellence 2023/2027 Project（卓越部门2023/2027项目）；Department of Information Engineering, Electrical Engineering and Applied Mathematics (DIEM), University of Salerno（信息工程、电气工程和应用数学系（DIEM），萨勒诺大学）；Specialization School in Oral Surgery, University of Naples Federico II（口腔外科专业学校，那不勒斯费德里科二世大学）
University of Leeds（利兹大学）；University College London（伦敦大学学院）
Human life is invaluable. When dangerous or life-threatening tasks need to be completed, robotic platforms could be ideal in replacing human operators. Such a task that we focus on in this work is the Explosive Ordnance Disposal. Robot telepresence has the potential to provide safety solutions, given that mobile robots have shown robust capabilities when operating in several environments. However, autonomy may be challenging and risky at this stage, compared to human operation. Teleoperation could be a compromise between full robot autonomy and human presence. In this paper, we present a relat…
Safety-Critical Bilateral Teleoperation for Omnidirectional Aerial Manipulation Using Force-Sensorless Haptic Feedback
Seoul National University（首尔大学）；Automation and System Research Institute (ASRI)（自动化与系统研究所）；Institute of Advanced Aerospace Technology (IAAT)（先进航空航天技术研究所）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Carnegie Mellon University（卡内基梅隆大学）
Bilateral teleoperation provides humanoid robots with human planning intelligence while enabling the human to feel what the robot feels. It has the potential to transform physically capable humanoid robots into dynamically intelligent ones. However, dynamic bilateral locomotion teleoperation remains as a challenge due to the complex dynamics it involves. This work presents our initial step to tackle this challenge via the concept of wheeled humanoid robot locomotion teleoperation by body tilt. Specifically, we developed a force-feedback-capable whole-body human-machine interface (HMI), and des…
Shared-Control Teleoperation Paradigms on a Soft Growing Robot Manipulator
Kadir Has University（卡迪尔哈斯大学）；University of Naples Federico II（那不勒斯费德里科二世大学）；Stanford University（斯坦福大学）；Washington State University（华盛顿州立大学）；Purdue University（普渡大学）；UC Santa Barbara（加州大学圣巴巴拉分校）
The tasks that an aerial manipulator can perform are incredibly diverse. However, nowadays the technology is not completely developed to achieve complex tasks autonomously. That's why we propose a human-in-the-loop system that can control a semi-autonomous aerial manipulator to accomplish these kinds of tasks. Furthermore, motivated by the growing trend of virtual reality systems, together with teleoperation, we develop a system composed of: an aerial manipulator model programmed in PX4 and modeled in Gazebo, a virtual reality immersion with an interactive controller, and the interconnection b…
