Camera-based tactile sensors have shown great promise in enhancing a robot's ability to perform a variety of dexterous manipulation tasks. Advantages of their use can be attributed to the high resolution tactile data and 3D depth map reconstructions they can provide. Unfortunately, many of these tactile sensors use either a flat sensing surface, sense on only one side of the sensor's body, or have a bulky form-factor, making it difficult to integrate the sensors with a variety of robotic grippers. Of the camera-based sensors that do have all-around, curved sensing surfaces, many cannot provide 3D depth maps; those that do often require optical designs specified to a particular sensor geometry. In this work, we introduce GelSight360, a fingertip-like, omnidirectional, camera-based tactile sensor capable of producing depth maps of objects deforming the sensor's surface. In addition, we introduce a novel cross-LED lighting scheme that can be implemented in different all-around sensor geometries and sizes, allowing the sensor to easily be reconfigured and attached to different grippers of varying DOFs. With this work, we enable roboticists to quickly and easily customize high resolution tactile sensors to fit their robotic system's needs.
Canonical Representation and Force-Based Pretraining of 3D Tactile for Dexterous Visuo-Tactile Policy Learning
Center on Frontiers of Computing Studies, School of Computer Science, Peking University（前沿计算研究中心，计算机科学学院，北京大学）；PKU-Agibot Lab, School of Computer Science, Peking University（北京大学计算机科学学院PKU-Agibot实验室）；National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（国家多媒体信息处理重点实验室，计算机科学学院，北京大学）
High resolution tactile sensors are often bulky and have shape profiles that make them awkward for use in manipulation. This becomes important when using such sensors as fingertips for dexterous multi-fingered hands, where boxy or planar fingertips limit the available set of smooth manipulation strategies. High resolution optical based sensors such as GelSight have until now been constrained to relatively flat geometries due to constraints on illumination geometry.Here, we show how to construct a rounded fingertip that utilizes a form of light piping for directional illumination. Our sensors c…
DexViTac: Collecting Human Visuo-Tactile-Kinematic Demonstrations for Contact-Rich Dexterous Manipulation
State Key Laboratory of Intelligent Manufacturing Equipment and Technology（智能制造装备与技术国家重点实验室）；Huazhong University of Science and Technology（华中科技大学）；Wuhan Huaweike Intelligent Technology Co., Ltd.（武汉华为凯科技有限公司）
Tactile sensing is an essential capability for robots that carry out dexterous manipulation tasks. While cameras, Lidars and other remote sensors can assess a scene globally and instantly, tactile sensors can reduce their measurement uncertainties and gain information about the local physical interactions between the in-contact objects and the robot, that are often not accessible via remote sensing. Tactile sensors can be grouped into two main categories: electronic tactile skins and camera based optical tactile sensors. The former are slim and can be fitted to different body parts, whereas th…
KineDex: Learning Tactile-Informed Visuomotor Policies via Kinesthetic Teaching for Dexterous Manipulation
Tongji University（同济大学）；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；University of Hong Kong（香港大学）；Shanghai Qi Zhi Institute（上海启智研究院）；Shanghai AI Lab（上海人工智能实验室）
University of Utah（犹他大学）；NVIDIA Corporation（英伟达公司）
While tactile skins have been shown to be useful for detecting collisions between a robotic arm and its environment, they have not been extensively used for improving robotic grasping and in-hand manipulation. We propose a novel sensor design for use in covering existing multi-fingered robot hands. We analyze the performance of four different piezoresistive materials using both fabric and anti-static foam substrates in benchtop experiments. We find that although the piezoresistive foam was designed as packing material and not for use as a sensing substrate, it performs comparably with fabrics …
Unified Visual-Tactile-Action Modeling from Human Demonstrations for Dexterous Manipulation
Shanghai Jiao Tong University（上海交通大学）；State Key Laboratory of General Artificial Intelligence, BIGAI（北京通用人工智能研究院 通用人工智能全国重点实验室）；Sharpa Robotics；Beijing Institute of Technology（北京理工大学）
FingerEye: Learning Dexterous Manipulation with Continuous Vision-Tactile Sensing
National University of Singapore（新加坡国立大学）；RoboScience（机器人科学）；Huazhong University of Science and Technology（华中科技大学）；South China University of Technology（华南理工大学）
Adaptive Visuo-Tactile Fusion with Predictive Force Attention for Dexterous Manipulation
Center on Frontiers of Computing Studies, School of Computer Science, Peking University（前沿计算研究中心，计算机科学学院，北京大学）；PKU-Agibot Lab, School of Computer Science, Peking University（PKU-Agibot实验室，计算机科学学院，北京大学）；National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机科学学院，北京大学）；Ant Group（蚂蚁集团）；Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）
Data-Driven Optimization of Tactile Sensor Configurations for Efficient Dexterous Manipulation
ShanghaiTech University, School of Information Science and Technology（上海科技大学信息科学与技术学院）；University of Alberta（阿尔伯塔大学）；Oklahoma State University（俄克拉荷马州立大学）；University of Colorado Denver, Department of Computer Science and Engineering（科罗拉多大学丹佛分校计算机科学与工程系）；Department of Robotics and Mechatronics Engineering, Kennesaw State University（凯斯西储大学机器人与机电工程系）
