Dextrous Tactile In-Hand Manipulation Using a Modular Reinforcement Learning Architecture
DLR Institute of Robotics and Mechatronics（DLR机器人与 mechatronics研究所）；Technical University of Munich（慕尼黑工业大学）；Deggendorf Institute of Technology（代根多夫应用技术大学）
Johannes Pitz, Lennart Röstel, Leon Sievers, Berthold Bäuml
Dextrous in-hand manipulation with a multi-fingered robotic hand is a challenging task, esp. when performed with the hand oriented upside down, demanding permanent force-closure, and when no external sensors are used. For the task of reorienting an object to a given goal orientation (vs. infinitely spinning it around an axis), the lack of external sensors is an additional fundamental challenge as the state of the object has to be estimated all the time, e.g., to detect when the goal is reached. In this paper, we show that the task of reorienting a cube to any of the 24 possible goal orientations in a $π$/2-raster using the torque-controlled DLR-Hand II is possible. The task is learned in simulation using a modular deep reinforcement learning architecture: the actual policy has only a small observation time window of 0.5s but gets the cube state as an explicit input which is estimated via a deep differentiable particle filter trained on data generated by running the policy. In simulation, we reach a success rate of 92% while applying significant domain randomization. Via zero-shot Sim2Real-transfer on the real robotic system, all 24 goal orientations can be reached with a high success rate.
Institute of Robotics and Mechatronics, German Aerospace Center (DLR)（德国航空航天中心（DLR）机器人与机电一体化研究所）；Deggendorf Institute of Technology（代根多夫应用技术大学）
We show that a purely tactile dextrous in-hand manipulation task with continuous regrasping, requiring permanent force closure, can be learned from scratch and executed robustly on a torque-controlled humanoid robotic hand. The task is rotating a cube without dropping it, but in contrast to OpenAI's seminal cube manipulation task, the palm faces downwards and no cameras but only the hand's position and torque sensing are used. Although the task seems simple, it combines for the first time all the challenges in execution as well as learning that are important for using in-hand manipulation in r…
Cross-Modal Visuo-Tactile Representation Learning with Action Chunking Transformers for Contact-Rich Manipulation
Guangdong Institute of Intelligence Science and Technology（广东智能科学与技术研究院）；Innovation and Research and Development Department, BoardWare Information System Company Ltd.（创新与研发部，BoardWare信息系统有限公司）；School of Artificial Intelligence, Nanjing Agricultural University（人工智能学院，南京农业大学）；School of Computing, National University of Singapore（计算机学院，新加坡国立大学）；School of Electronic Engineering and Automation, Guilin University of Electronic Technology（电子工程与自动化学院，桂林电子科技大学）
AetheRock: An Arm-Worn Robot Teaching System for Force-Guided Vision-Tactile Learning
Shanghai Jiao Tong University（上海交通大学）；Ant Group（蚂蚁集团）；Shanghai Innovation Institute（上海创新研究院）；Beijing Institute for General Artificial Intelligence (BIGAI)（北京通用人工智能研究院）
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Hong Kong University of Science and Technology（香港科技大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深港协同创新研究院）
Tactile sensing is an essential perception for robots to complete dexterous tasks. As a promising tactile sensing technique, vision-based tactile sensors have been developed to improve robot performance in manipulation and grasping. Here we propose a new design of a vision-based tactile sensor, DelTact. The sensor uses a modular hardware architecture for compactness whilst maintaining a contact measurement of full resolution (798*586) and large area (675mm2). Moreover, it adopts an improved dense random color pattern based on the previous version to achieve high accuracy of contact deformation…
MISTac: A Vision-Based Tactile Sensor for Minimally Invasive Surgery
TUD Dresden University of Technology（德累斯顿工业大学）；Stanford University（斯坦福大学）；University Hospital Carl Gustav Carus（卡尔·古斯塔夫·卡鲁斯大学医院）；Centre for Tactile Internet with Human-in-the-Loop (CeTI)（以人为中心的触觉互联网中心）；National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心）；DKFZ（德国癌症研究中心）；Helmholtz-Zentrum Dresden-Rossendorf (HZDR)（德累斯顿-罗森多夫亥姆霍兹中心）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
External Photoreflective Tactile Sensing Based on Surface Deformation Measurement
Graduate School of Engineering Science, The University of Osaka（大阪大学工学研究科）；Department of Mechanical Engineering and Science, Kyoto University（京都大学机械工学科）；Faculty of Design Department of Media Design, Kyushu University（九州大学设计学部媒体设计学科）
This paper describes the design of a multi-camera optical tactile sensor that provides information about the contact force distribution applied to its soft surface. This information is contained in the motion of spherical particles spread within the surface, which deforms when subject to force. The small embedded cameras capture images of the different particle patterns that are then mapped to the three-dimensional contact force distribution through a machine learning architecture. The design proposed in this paper exhibits a larger contact surface and a thinner structure than most of the exis…
Massachusetts Institute of Technology（麻省理工学院）；Lightspeed & Quantum Studios, Tencent America（腾讯美国光速量子工作室）
Traditional robotic manipulator design methods require extensive, time-consuming, and manual trial and error to produce a viable design. During this process, engineers often spend their time redesigning or reshaping components as they discover better topologies for the robotic manipulator. Tactile sensors, while useful, often complicate the design due to their bulky form factor. We propose an integrated design pipeline to streamline the design and manufacturing of robotic manipulators with knitted, glove-like tactile sensors. The proposed pipeline allows a designer to assemble a collection of …
Facebook AI Research（脸书人工智能研究院）；University of California, Berkeley（加州大学伯克利分校）；Shanghai Jiao Tong University（上海交通大学）；Massachusetts Institute of Technology（麻省理工学院）
With the increased availability of rich tactile sensors, there is an equally proportional need for open-source and integrated software capable of efficiently and effectively processing raw touch measurements into high-level signals that can be used for control and decision-making. In this paper, we present PyTouch -- the first machine learning library dedicated to the processing of touch sensing signals. PyTouch, is designed to be modular, easy-to-use and provides state-of-the-art touch processing capabilities as a service with the goal of unifying the tactile sensing community by providing a …
SoFiE: Soft Finger Exoskeleton for Intelligent Grasping
SDU Soft Robotics, SDU Biorobotics, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark (SDU)（SDU柔性机器人实验室、SDU生物机器人实验室、马士基麦金尼莫勒研究所、南丹麦大学）
