This report presents the experimental methodology and a step-by-step guide for gathering data on how aging influences tactile surface perception in decision and action. The experiments consist of a set of trials in which the ability to distinguish tactile stimuli is investigated. A robot arm is used to provide a systematic and unbiased presentation of the stimuli.
University of Bristol（布里斯托大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）
This article describes a new way of controlling robots using soft tactile sensors: pose-based tactile servo (PBTS) control. The basic idea is to embed a tactile perception model for estimating the sensor pose within a servo control loop that is applied to local object features such as edges and surfaces. PBTS control is implemented with a soft curved optical tactile sensor (the BRL TacTip) using a convolutional neural network trained to be insensitive to shear. In consequence, robust and accurate controlled motion over various complex 3D objects is attained. First, we review tactile servoing a…
University of Bristol（布里斯托大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）
Deep learning combined with high-resolution tactile sensing could lead to highly capable dexterous robots. However, progress is slow because of the specialist equipment and expertise. The DIGIT tactile sensor offers low-cost entry to high-resolution touch using GelSight-type sensors. Here we customize the DIGIT to have a 3D-printed sensing surface based on the TacTip family of soft biomimetic optical tactile sensors. The DIGIT-TacTip (DigiTac) enables direct comparison between these distinct tactile sensor types. For this comparison, we introduce a tactile robot system comprising a desktop arm…
We present MidasTouch, a tactile perception system for online global localization of a vision-based touch sensor sliding on an object surface. This framework takes in posed tactile images over time, and outputs an evolving distribution of sensor pose on the object's surface, without the need for visual priors. Our key insight is to estimate local surface geometry with tactile sensing, learn a compact representation for it, and disambiguate these signals over a long time horizon. The backbone of MidasTouch is a Monte-Carlo particle filter, with a measurement model based on a tactile code networ…
Bristol Robotics Laboratory（布里斯托机器人实验室）；University of Bristol（布里斯托大学）
Our human sense of touch enables us to manipulate our surroundings; therefore, complex robotic manipulation will require artificial tactile sensing. Typically tactile sensor arrays are used in robotics, implying that a straightforward way of interpreting multidimensional data is required. In this paper we present a simple visualisation approach based on applying principal component analysis (PCA) to systematically collected sets of tactile data. We apply the visualisation approach to 4 different types of tactile sensor, encompassing fingertips and vibrissal arrays. The results show that PCA ca…
TouchGuide: Inference-Time Steering of Visuomotor Policies via Touch Guidance
Shanghai Jiao Tong University（上海交通大学）；Xense Robotics（Xense机器人公司）；Sun Yat-sen University（中山大学）；Oxford（牛津大学）；Shanghai AI Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）；UCSD（加州大学圣地亚哥分校）
Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention
Ant Group（蚂蚁集团）；Zhejiang University（浙江大学）；The University of Hong Kong（香港大学）；Shenzhen Loop Area Institute（深圳河套学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
University of Bristol（布里斯托大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）
High-resolution optical tactile sensors are increasingly used in robotic learning environments due to their ability to capture large amounts of data directly relating to agent-environment interaction. However, there is a high barrier of entry to research in this area due to the high cost of tactile robot platforms, specialised simulation software, and sim-to-real methods that lack generality across different sensors. In this letter we extend the Tactile Gym simulator to include three new optical tactile sensors (TacTip, DIGIT and DigiTac) of the two most popular types, Gelsight-style (image-sh…
Model-Based Capacitive Touch Sensing in Soft Robotics: Achieving Robust Tactile Interactions for Artistic Applications
Departamento de Mecánica de la Universidad de Chile（智利大学机械系）；Soft Robotics Lab (SRL) at ETH Zurich（苏黎世联邦理工学院软机器人实验室）；University of Applied Sciences Karlsruhe（卡尔斯鲁厄应用技术大学）；Proxception GmbH（Proxception有限公司）
This work investigates uncertainty-aware deep learning (DL) in tactile robotics based on a general framework introduced recently for robot vision. For a test scenario, we consider optical tactile sensing in combination with DL to estimate the edge pose as a feedback signal to servo around various 2D test objects. We demonstrate that uncertainty-aware DL can improve the pose estimation over deterministic DL methods. The system estimates the uncertainty associated with each prediction, which is used along with temporal coherency to improve the predictions via a Kalman filter, and hence improve t…
For robot touch to converge with the human sense of touch, artificial transduction should involve biologically-plausible population codes analogous to those of natural afferents. Using a biomimetic tactile sensor with 3d-printed skin based on the dermal-epidermal boundary, we propose two novel feature sets to mimic slowly-adapting and rapidly-adapting type-I tactile mechanoreceptor function. Their plausibility is tested with three classic experiments from the study of natural touch: impingement on a flat plate to probe adaptation and spatial modulation; stimulation by spatially-complex ridged …
TacGen: Touch Is a Necessary Dimension of Physical-World Representation -- Addressing Tactile Data Scarcity with Scalable Vision-to-Touch Alignment and Generation
University of Maryland, College Park（马里兰大学帕克分校）；Carnegie Mellon University（卡内基梅隆大学）；University of Alabama（阿拉巴马大学）；University of Washington（华盛顿大学）；Stanford University（斯坦福大学）
ViTacGen: Robotic Pushing with Vision-to-Touch Generation
Department of Engineering, King’s College London（工程系，伦敦国王学院）；Department of Engineering Mathematics and Bristol, Robotics Laboratory, University of Bristol（工程数学系及布里斯托尔机器人实验室，布里斯托尔大学）
Koc University（科奇大学）；University of Lille（里尔大学）；École de Technologie Supérieure（高等技术学院）；Pohang University of Science and Technology (POSTECH)（浦项科技大学（POSTECH））
We review the current technology underlying surface haptics that converts passive touch surfaces to active ones (machine haptics), our perception of tactile stimuli displayed through active touch surfaces (human haptics), their potential applications (human-machine interaction), and finally the challenges ahead of us in making them available through commercial systems. This review primarily covers the tactile interactions of human fingers or hands with surface-haptics displays by focusing on the three most popular actuation methods: vibrotactile, electrostatic, and ultrasonic.
Let Robots Feel Your Touch: Visuo-Tactile Cortical Alignment for Embodied Mirror Resonance
Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Key Laboratory of Biomedical Engineering of Hainan Province, School of Biomedical Engineering, Hainan University（海南省生物医学工程重点实验室，海南大学生物医学工程学院）；School of New Media Art and Design, Beihang University（北航艺术与设计学院）；MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology（教育部生物医学光子学重点实验室，武汉光电研究所，华中科技大学）
Display of illusory vibration locations between physical vibrotactile motors (tactors) placed on the skin has the potential to reduce the number of tactors in distributed tactile displays. This paper presents a between-tactor display method that uses dynamic tactile stimuli to generate illusory vibration locations. A belt with only 6 vibration motors displays 24 targets consisting of on-tactor and between-tactor locations. On-tactor locations are represented by simply vibrating the relevant single tactor. Between-tactor locations are displayed by adjusting the relative vibration amplitudes of …
Carnegie Mellon University（卡内基梅隆大学）；Brigham Young University（杨百翰大学）
Knowledge of 3-D object shape is of great importance to robot manipulation tasks, but may not be readily available in unstructured environments. While vision is often occluded during robot-object interaction, high-resolution tactile sensors can give a dense local perspective of the object. However, tactile sensors have limited sensing area and the shape representation must faithfully approximate non-contact areas. In addition, a key challenge is efficiently incorporating these dense tactile measurements into a 3-D mapping framework. In this work, we propose an incremental shape mapping method …
Inspired by humans' ability to perceive the surface texture of unfamiliar objects without relying on vision, the sense of touch can play a crucial role in robots exploring the environment, particularly in scenes where vision is difficult to apply, or occlusion is inevitable. Existing tactile surface reconstruction methods rely on external sensors or have strong prior assumptions, making the operation complex and limiting their application scenarios. This paper presents a framework for low-drift surface reconstruction through multiple tactile measurements, Tac2Structure. Compared with existing …
University of Liverpool（利物浦大学）；King’s College London（伦敦国王学院）；Monash University（莫纳什大学）
Picking up transparent objects is still a challenging task for robots. The visual properties of transparent objects such as reflection and refraction make the current grasping methods that rely on camera sensing fail to detect and localise them. However, humans can handle the transparent object well by first observing its coarse profile and then poking an area of interest to get a fine profile for grasping. Inspired by this, we propose a novel framework of vision-guided tactile poking for transparent objects grasping. In the proposed framework, a segmentation network is first used to predict t…
University of Bristol（布里斯托大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）
This article illustrates the application of deep learning to robot touch by considering a basic yet fundamental capability: estimating the relative pose of part of an object in contact with a tactile sensor. We begin by surveying deep learning applied to tactile robotics, focussing on optical tactile sensors, which help bridge from deep learning for vision to touch. We then show how deep learning can be used to train accurate pose models of 3D surfaces and edges that are insensitive to nuisance variables such as motion-dependent shear. This involves including representative motions as unlabell…
University of Bristol（布里斯托大学）；Universit di Pisa（比萨大学）；Istituto Italiano di Tecnologia（意大利理工学院）
In this work, we report on the integrated sensorimotor control of the Pisa/IIT SoftHand, an anthropomorphic soft robot hand designed around the principle of adaptive synergies, with the BRL tactile fingertip (TacTip), a soft biomimetic optical tactile sensor based on the human sense of touch. Our focus is how a sense of touch can be used to control an anthropomorphic hand with one degree of actuation, based on an integration that respects the hand's mechanical functionality. We consider: (i) closed-loop tactile control to establish a light contact on an unknown held object, based on the struct…
University of Bristol（布里斯托大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）
Reproducing the capabilities of the human sense of touch in machines is an important step in enabling robot manipulation to have the ease of human dexterity. A combination of robotic technologies will be needed, including soft robotics, biomimetics and the high-resolution sensing offered by optical tactile sensors. This combination is considered here as a SoftBOT (Soft Biomimetic Optical Tactile) sensor. This article reviews the BRL TacTip as a prototypical example of such a sensor. Topics include the relation between artificial skin morphology and the transduction principles of human touch, t…
University of California, Santa Barbara（加州大学圣巴巴拉分校）
Tactile sensing is a essential for skilled manipulation and object perception, but existing devices are unable to capture mechanical signals in the full gamut of regimes that are important for human touch sensing, and are unable to emulate the sensing abilities of the human hand. Recent research reveals that human touch sensing relies on the transmission of mechanical waves throughout tissues of the hand. This provides the hand with remarkable abilities to remotely capture distributed vibration signatures of touch contact. Little engineering attention has been given to important sensory system…
Tactile afferents such as (RA), and Pacinian (PC) afferents that respond to external stimuli enable complicated actions such as grasping, stroking and identifying an object. To understand the tactile sensation induced by these actions deeply, the activities of the tactile afferents need to be revealed. For this purpose, we develop a computational model for each tactile afferent for vibration stimuli, combining finite element analysis finite element method (FEM) analysis and a leaky integrate-and-fire model that represents the neural characteristics. This computational model can easily estimate…
King’s College London（伦敦国王学院）；University of Liverpool（利物浦大学）；Queen Mary University of London（伦敦玛丽女王大学）
For humans, both the proprioception and touch sensing are highly utilized when performing haptic perception. However, most approaches in robotics use only either proprioceptive data or touch data in haptic object recognition. In this paper, we present a novel method named Iterative Closest Labeled Point (iCLAP) to link the kinesthetic cues and tactile patterns fundamentally and also introduce its extensions to recognize object shapes. In the training phase, the iCLAP first clusters the features of tactile readings into a codebook and assigns these features with distinct label numbers. A 4D poi…
Look-to-Touch: A Vision-Enhanced Proximity and Tactile Sensor for Distance and Geometry Perception in Robotic Manipulation
State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University（上海交通大学机械系统与振动国家重点实验室）；Robotics Institute, School of Mechanical Engineering, Shanghai Jiao Tong University（上海交通大学机械工程学院机器人研究所）
In this paper, we present an approach to tactile pose estimation from the first touch for known objects. First, we create an object-agnostic map from real tactile observations to contact shapes. Next, for a new object with known geometry, we learn a tailored perception model completely in simulation. To do so, we simulate the contact shapes that a dense set of object poses would produce on the sensor. Then, given a new contact shape obtained from the sensor output, we match it against the pre-computed set using the object-specific embedding learned purely in simulation using contrastive learni…
Embedding high-resolution touch across robotic hands enables adaptive human-like grasping
Institute for Artificial Intelligence（人工智能研究所）；Peking University（北京大学）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；PKU-Wuhan Institute for Artificial Intelligence（北大武汉人工智能研究所）；College of Engineering（工程学院）；School of Engineering and Materials Science（工程与材料科学学院）
