University of Bristol（布里斯托大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）
Nathan F. Lepora, John Lloyd
备注：
Accepted in IEEE Robotics & Automation Magazine Special Issue on Deep Learning and Machine Learning in Robotics. NL and JL contributed equally to this work
This article illustrates the application of deep learning to robot touch by considering a basic yet fundamental capability: estimating the relative pose of part of an object in contact with a tactile sensor. We begin by surveying deep learning applied to tactile robotics, focussing on optical tactile sensors, which help bridge from deep learning for vision to touch. We then show how deep learning can be used to train accurate pose models of 3D surfaces and edges that are insensitive to nuisance variables such as motion-dependent shear. This involves including representative motions as unlabelled perturbations of the training data and using Bayesian optimization of the network and training hyperparameters to find the most accurate models. Accurate estimation of pose from touch will enable robots to safely and precisely control their physical interactions, underlying a wide range of object exploration and manipulation tasks.
Model-Based Capacitive Touch Sensing in Soft Robotics: Achieving Robust Tactile Interactions for Artistic Applications
Departamento de Mecánica de la Universidad de Chile（智利大学机械系）；Soft Robotics Lab (SRL) at ETH Zurich（苏黎世联邦理工学院软机器人实验室）；University of Applied Sciences Karlsruhe（卡尔斯鲁厄应用技术大学）；Proxception GmbH（Proxception有限公司）
University of Bristol（布里斯托大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）
Deep learning combined with high-resolution tactile sensing could lead to highly capable dexterous robots. However, progress is slow because of the specialist equipment and expertise. The DIGIT tactile sensor offers low-cost entry to high-resolution touch using GelSight-type sensors. Here we customize the DIGIT to have a 3D-printed sensing surface based on the TacTip family of soft biomimetic optical tactile sensors. The DIGIT-TacTip (DigiTac) enables direct comparison between these distinct tactile sensor types. For this comparison, we introduce a tactile robot system comprising a desktop arm…
Embedding high-resolution touch across robotic hands enables adaptive human-like grasping
Institute for Artificial Intelligence（人工智能研究所）；Peking University（北京大学）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；PKU-Wuhan Institute for Artificial Intelligence（北大武汉人工智能研究所）；College of Engineering（工程学院）；School of Engineering and Materials Science（工程与材料科学学院）
University of Bristol（布里斯托大学）；Universit di Pisa（比萨大学）；Istituto Italiano di Tecnologia（意大利理工学院）
In this work, we report on the integrated sensorimotor control of the Pisa/IIT SoftHand, an anthropomorphic soft robot hand designed around the principle of adaptive synergies, with the BRL tactile fingertip (TacTip), a soft biomimetic optical tactile sensor based on the human sense of touch. Our focus is how a sense of touch can be used to control an anthropomorphic hand with one degree of actuation, based on an integration that respects the hand's mechanical functionality. We consider: (i) closed-loop tactile control to establish a light contact on an unknown held object, based on the struct…
University of Bristol（布里斯托大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）
This article describes a new way of controlling robots using soft tactile sensors: pose-based tactile servo (PBTS) control. The basic idea is to embed a tactile perception model for estimating the sensor pose within a servo control loop that is applied to local object features such as edges and surfaces. PBTS control is implemented with a soft curved optical tactile sensor (the BRL TacTip) using a convolutional neural network trained to be insensitive to shear. In consequence, robust and accurate controlled motion over various complex 3D objects is attained. First, we review tactile servoing a…
When Vision Meets Touch: A Contemporary Review for Visuotactile Sensors from the Signal Processing Perspective
Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；RISC-V International Open Source Laboratory（RISC-V国际开源实验室）；National University of Singapore（新加坡国立大学）；Tsinghua University（清华大学）
Distributed sensor arrays capable of detecting multiple spatially distributed stimuli are considered an important element in the realisation of exteroceptive and proprioceptive soft robots. This paper expands upon the previously presented idea of decoupling the measurements of pressure and location of a local indentation from global deformation, using the overall stretch experienced by a soft capacitive e-skin. We employed machine learning methods to decouple and predict these highly coupled deformation stimuli, collecting data from a soft sensor e-skin which was then fed to a machine learning…
University of Bristol（布里斯托大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）
Reproducing the capabilities of the human sense of touch in machines is an important step in enabling robot manipulation to have the ease of human dexterity. A combination of robotic technologies will be needed, including soft robotics, biomimetics and the high-resolution sensing offered by optical tactile sensors. This combination is considered here as a SoftBOT (Soft Biomimetic Optical Tactile) sensor. This article reviews the BRL TacTip as a prototypical example of such a sensor. Topics include the relation between artificial skin morphology and the transduction principles of human touch, t…
Stanford University（斯坦福大学）；Reality Labs Research, Meta Platforms Inc.（元平台公司现实实验室研究部）
When humans socially interact with another agent (e.g., human, pet, or robot) through touch, they do so by applying varying amounts of force with different directions, locations, contact areas, and durations. While previous work on touch gesture recognition has focused on the spatio-temporal distribution of normal forces, we hypothesize that the addition of shear forces will permit more reliable classification. We present a soft, flexible skin with an array of tri-axial tactile sensors for the arm of a person or robot. We use it to collect data on 13 touch gesture classes through user studies …
Humans represent and discriminate the objects in the same category using their properties, and an intelligent robot should be able to do the same. In this paper, we build a robot system that can autonomously perceive the object properties through touch. We work on the common object category of clothing. The robot moves under the guidance of an external Kinect sensor, and squeezes the clothes with a GelSight tactile sensor, then it recognizes the 11 properties of the clothing according to the tactile data. Those properties include the physical properties, like thickness, fuzziness, softness and…
Czech Technical University in Prague（布拉格捷克理工大学）；Czech Institute of Informatics, Robotics, and Cybernetics, Czech Technical University in Prague（布拉格捷克理工大学捷克信息学、机器人学与控制论研究所）
Mechanism calibration is an important and non-trivial task in robotics. Advances in sensor technology make affordable but increasingly accurate devices such as cameras and tactile sensors available, making it possible to perform automated self-contained calibration relying on redundant information in these sensory streams. In this work, we use a simulated iCub humanoid robot with a stereo camera system and end-effector contact emulation to quantitatively compare the performance of kinematic calibration by employing different combinations of intersecting kinematic chains -- either through self-…
Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente（电子工程、数学和计算机科学学院，代尔夫特理工大学）；Department of Electrical Engineering, Wright State University（电气工程系，怀特州立大学）
Mapping Embodied Affective Touch Strategies on a Humanoid Robot
Faculty of Engineering and Architecture（工程与建筑学院）；IDLab-AIRO, Ghent University – imec（IDLab-AIRO，根特大学 – imec）；Italian Institute of Technology（意大利理工学院）；School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology（电气工程与计算机科学学院，皇家理工学院）
Multi-modal perception for soft robotic interactions using generative models
The BioRobotics Institute, Sant’Anna School of Advanced Studies（圣安娜高级研究院生物机器人研究所）；Sant’Anna School of Advanced Studies（圣安娜高级研究院）；University College London（伦敦大学学院）
Humans perceive the world using multi-modal sensory inputs such as vision, audition, and touch. In this work, we investigate the cross-modal connection between vision and touch. The main challenge in this cross-domain modeling task lies in the significant scale discrepancy between the two: while our eyes perceive an entire visual scene at once, humans can only feel a small region of an object at any given moment. To connect vision and touch, we introduce new tasks of synthesizing plausible tactile signals from visual inputs as well as imagining how we interact with objects given tactile data a…
Nara Institute of Science and Technology（奈良先端科学技术大学院大学）
Affective touch offers positive psychological and physiological benefits such as the mitigation of stress and pain. If a robot could realize human-like affective touch, it would open up new application areas, including supporting care work. In this research, we focused on the gentle stroking motion of a robot to evoke the same emotions that human touch would evoke: in other words, an affective touch robot. We propose a robot that is able to gently stroke the back of a human using our designed human-imitation hand. To evaluate the emotional effects of this affective touch, we compared the resul…
Massachusetts Institute of Technology（麻省理工学院）；Meta AI
Simulators perform an important role in prototyping, debugging, and benchmarking new advances in robotics and learning for control. Although many physics engines exist, some aspects of the real world are harder than others to simulate. One of the aspects that have so far eluded accurate simulation is touch sensing. To address this gap, we present TACTO - a fast, flexible, and open-source simulator for vision-based tactile sensors. This simulator allows to render realistic high-resolution touch readings at hundreds of frames per second, and can be easily configured to simulate different vision-…
Despite significant advances in touch and force transduction, tactile sensing is still far from ubiquitous in robotic manipulation. Existing methods for building touch sensors have proven difficult to integrate into robot fingers due to multiple challenges, including difficulty in covering multicurved surfaces, high wire count, or packaging constrains preventing their use in dexterous hands. In this paper, we present a multicurved robotic finger with accurate touch localization and normal force detection over complex, three-dimensional surfaces. The key to our approach is the novel use of over…
The sense of touch is fundamental in several manipulation tasks, but rarely used in robot manipulation. In this work we tackle the problem of learning rich touch features from cross-modal self-supervision. We evaluate them identifying objects and their properties in a few-shot classification setting. Two new datasets are introduced using a simulated anthropomorphic robotic hand equipped with tactile sensors on both synthetic and daily life objects. Several self-supervised learning methods are benchmarked on these datasets, by evaluating few-shot classification on unseen objects and poses. Our …
PAL Robotics S.L.（PAL机器人有限公司）；Bielefeld University（比勒费尔德大学）
Despite many advances in service robotics, successful and secure object manipulation on mobile platforms is still a challenge. In order to come closer to human grasping performance, it is natural to provide robots with the same capability that humans have: the sense of touch. This abstract presents novel, tactile-equipped end-effectors for the service robot TIAGo that are currently being developed. Their primary goal is to improve reliability and success of mobile manipulation, but they also enable further research in related fields such as learning by human demonstration, object exploration a…
Università di Bergamo（贝加莫大学）；Czech Technical University in Prague（捷克布拉格技术大学）；Comenius University Bratislava（布拉迪斯拉发夸美纽斯大学）；Central European University（中欧大学）
Touch is a key part of interaction and communication between humans, but has still been little explored in human-robot interaction. In this work, participants were asked to approach and touch a humanoid robot on the hand (Nao - 26 participants; Pepper - 28 participants) to get its attention. We designed reaction behaviors for the robot that consisted in four different combinations of arm movements with the touched hand moving forward or back and the other hand moving forward or staying in place, with simultaneous leaning back, followed by looking at the participant. We studied which reaction o…
ViTacGen: Robotic Pushing with Vision-to-Touch Generation
Department of Engineering, King’s College London（工程系，伦敦国王学院）；Department of Engineering Mathematics and Bristol, Robotics Laboratory, University of Bristol（工程数学系及布里斯托尔机器人实验室，布里斯托尔大学）
Touch2Robot: Robot Touch in the Human Demonstration Loop
Beijing Institute for General Artificial Intelligence (BIGAI)（北京通用人工智能研究院）；ShanghaiTech University（上海科技大学）；Shanghai Jiao Tong University（上海交通大学）；Beihang University（北京航空航天大学）
This work investigates uncertainty-aware deep learning (DL) in tactile robotics based on a general framework introduced recently for robot vision. For a test scenario, we consider optical tactile sensing in combination with DL to estimate the edge pose as a feedback signal to servo around various 2D test objects. We demonstrate that uncertainty-aware DL can improve the pose estimation over deterministic DL methods. The system estimates the uncertainty associated with each prediction, which is used along with temporal coherency to improve the predictions via a Kalman filter, and hence improve t…
For certain manipulation tasks, object pose estimation from head-mounted cameras may not be sufficiently accurate. This is at least in part due to our inability to perfectly calibrate the coordinate frames of today's high degree of freedom robot arms that link the head to the end-effectors. We present a novel framework combining pre-touch sensing and deep learning to more accurately estimate pose in an efficient manner. The use of pre-touch sensing allows our method to localize the object directly with respect to the robot's end effector, thereby avoiding error caused by miscalibration of the …
University of Bristol（布里斯托大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）
High-resolution optical tactile sensors are increasingly used in robotic learning environments due to their ability to capture large amounts of data directly relating to agent-environment interaction. However, there is a high barrier of entry to research in this area due to the high cost of tactile robot platforms, specialised simulation software, and sim-to-real methods that lack generality across different sensors. In this letter we extend the Tactile Gym simulator to include three new optical tactile sensors (TacTip, DIGIT and DigiTac) of the two most popular types, Gelsight-style (image-sh…
Let Robots Feel Your Touch: Visuo-Tactile Cortical Alignment for Embodied Mirror Resonance
Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Key Laboratory of Biomedical Engineering of Hainan Province, School of Biomedical Engineering, Hainan University（海南省生物医学工程重点实验室，海南大学生物医学工程学院）；School of New Media Art and Design, Beihang University（北航艺术与设计学院）；MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology（教育部生物医学光子学重点实验室，武汉光电研究所，华中科技大学）
Touch and Tell: Multimodal Decoding of Human Emotions and Social Gestures for Robots
Faculty of Engineering and Architecture（工程与建筑学院）；IDLab-AIRO, Ghent University – imec（IDLab-AIRO，根特大学–imec）；Department of Engineering Science, University of Oxford（工程科学系，牛津大学）
Interacting with humans is one of the main challenges for mobile robots in a human inhabited environment. To enable adaptive behavior, a robot needs to recognize touch gestures and/or the proximity to interacting individuals. Moreover, a robot interacting with two or more humans usually needs to distinguish between them. However, this remains both a configuration and cost intensive task. In this paper we utilize inexpensive Bluetooth Low Energy (BLE) devices and propose an easy and configurable technique to enhance the robot's capabilities to interact with surrounding people. In a noisy labora…
For robot touch to converge with the human sense of touch, artificial transduction should involve biologically-plausible population codes analogous to those of natural afferents. Using a biomimetic tactile sensor with 3d-printed skin based on the dermal-epidermal boundary, we propose two novel feature sets to mimic slowly-adapting and rapidly-adapting type-I tactile mechanoreceptor function. Their plausibility is tested with three classic experiments from the study of natural touch: impingement on a flat plate to probe adaptation and spatial modulation; stimulation by spatially-complex ridged …
