Magnetic-based tactile sensors (MBTS) combine the advantages of compact design and high-frequency operation but suffer from limited spatial resolution due to their sparse taxel arrays. This paper proposes SuperMag, a tactile shape reconstruction method that addresses this limitation by leveraging high-resolution vision-based tactile sensor (VBTS) data to supervise MBTS super-resolution. Co-designed, open-source VBTS and MBTS with identical contact modules enable synchronized data collection of high-resolution shapes and magnetic signals via a symmetric calibration setup. We frame tactile shape reconstruction as a conditional generative problem, employing a conditional variational auto-encoder to infer high-resolution shapes from low-resolution MBTS inputs. The MBTS achieves a sampling frequency of 125 Hz, whereas the shape reconstruction sustains an inference time within 2.5 ms. This cross-modality synergy advances tactile perception of the MBTS, potentially unlocking its new capabilities in high-precision robotic tasks.
University of Liverpool（利物浦大学）；King’s College London（伦敦国王学院）；Monash University（莫纳什大学）
Picking up transparent objects is still a challenging task for robots. The visual properties of transparent objects such as reflection and refraction make the current grasping methods that rely on camera sensing fail to detect and localise them. However, humans can handle the transparent object well by first observing its coarse profile and then poking an area of interest to get a fine profile for grasping. Inspired by this, we propose a novel framework of vision-guided tactile poking for transparent objects grasping. In the proposed framework, a segmentation network is first used to predict t…
Crack detection is of great significance for monitoring the integrity and well-being of the infrastructure such as bridges and underground pipelines, which are harsh environments for people to access. In recent years, computer vision techniques have been applied in detecting cracks in concrete structures. However, they suffer from variances in light conditions and shadows, lacking robustness and resulting in many false positives. To address the uncertainty in vision, human inspectors actively touch the surface of the structures, guided by vision, which has not been explored in autonomous crack…
BMW Group（宝马集团）；University of Glasgow（格拉斯哥大学）；Technische Universität München（慕尼黑工业大学）；Donders Institute for Brain and Cognition（唐德斯脑与认知研究所）；Radboud University（拉德堡德大学）
This paper proposes a novel active visuo-tactile based methodology wherein the accurate estimation of the time-invariant SE(3) pose of objects is considered for autonomous robotic manipulators. The robot equipped with tactile sensors on the gripper is guided by a vision estimate to actively explore and localize the objects in the unknown workspace. The robot is capable of reasoning over multiple potential actions, and execute the action to maximize information gain to update the current belief of the object. We formulate the pose estimation process as a linear translation invariant quaternion …
Traditional methods to achieve high localization accuracy with tactile sensors usually use a matrix of miniaturized individual sensors distributed on the area of interest. This approach usually comes at a price of increased complexity in fabrication and circuitry, and can be hard to adapt for non planar geometries. We propose to use low cost optic components mounted on the edges of the sensing area to measure how light traveling through an elastomer is affected by touch. Multiple light emitters and receivers provide us with a rich signal set that contains the necessary information to pinpoint …
MVP-Tac: A Miniaturized Dual-Modal Vision and Photoelastic Tactile Sensor for Robot-Assisted Minimally Invasive Surgery
Elmore Family School of Electrical and Computer Engineering, Purdue University（普渡大学埃尔莫尔电气与计算机工程学院）；Edwardson School of Industrial Engineering, Purdue University（普渡大学爱德华森工业工程学院）；School of Mechanical Engineering, Purdue University（普渡大学机械工程学院）；Robotics Engineering Technology, Purdue University（普渡大学机器人工程技术学院）；Department of Civil Engineering, Stony Brook University（石溪大学土木工程系）；Department of Mechanical Engineering, Michigan State University（密歇根州立大学机械工程系）
MagCilia: A Compact Magnetociliary Tactile Sensor with 3D Force Sensing for Robotic Contact Perception and Grasping Feedback
National University of Singapore（新加坡国立大学）；University of California, Berkeley（加州大学伯克利分校）；City University of Hong Kong（香港城市大学）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）
TacPrint: A Wearable Fingertip Tactile Sensor for Human-to-Robot Contact Reproduction
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Imprintx Robotics（Imprintx机器人公司）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）
TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation
TARS Robotics；National University of Singapore（新加坡国立大学）；Shanghai Jiao Tong University（上海交通大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Fudan University（复旦大学）
TacEva: A Performance Evaluation Framework For Vision-Based Tactile Sensors
Department of Bioengineering, Imperial College London（帝国理工学院生物工程系）；Waseda University（早稻田大学）；Department of Engineering, King’s College London（伦敦国王学院工程系）；Centre for Advanced Robotics, Queen Mary University of London（伦敦女王学院先进机器人中心）
AetheRock: An Arm-Worn Robot Teaching System for Force-Guided Vision-Tactile Learning
Shanghai Jiao Tong University（上海交通大学）；Ant Group（蚂蚁集团）；Shanghai Innovation Institute（上海创新研究院）；Beijing Institute for General Artificial Intelligence (BIGAI)（北京通用人工智能研究院）
Manipulating cables is challenging for robots because of the infinite degrees of freedom of the cables and frequent occlusion by the gripper and the environment. These challenges are further complicated by the dexterous nature of the operations required for cable routing and assembly, such as weaving and inserting, hampering common solutions with vision-only sensing. In this paper, we propose to integrate tactile-guided low-level motion control with high-level vision-based task parsing for a challenging task: cable routing and assembly on a reconfigurable task board. Specifically, we build a l…
TORL-VLA: Tactile Guided Online Reinforcement Learning for Contact-Rich Manipulation
Meituan（美团）；Beijing Institute of Technology（北京理工大学）；Beihang University（北京航空航天大学）；State Key Lab of Multimodal Artificial Intelligence Systems, Institute of Automation, CAS（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；China University of Mining and Technology (Beijing)（中国矿业大学（北京））
Massachusetts Institute of Technology（麻省理工学院）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
针对柔性结构与高分辨率触觉感知协同设计的难题，提出融合有限元力学仿真与PBR光学仿真的GelSight Fin Ray传感器端到端设计框架，可优化多类设计参数，缩短原型周期，验证日常物品抓取效用。
The sense of touch plays a key role in enabling humans to understand and interact with surrounding environments. For robots, tactile sensing is also irreplaceable. While interacting with objects, tactile sensing provides useful information for the robot to understand the object, such as distributed pressure, temperature, vibrations and texture. During robot grasping, vision is often occluded by its end-effectors, whereas tactile sensing can measure areas that are not accessible by vision. In the past decades, a number of tactile sensors have been developed for robots and used for different rob…
This work studies the problem of shape reconstruction and object localization using a vision-based tactile sensor, GelSlim. The main contributions are the recovery of local shapes from contact, an approach to reconstruct the tactile shape of objects from tactile imprints, and an accurate method for object localization of previously reconstructed objects. The algorithms can be applied to a large variety of 3D objects and provide accurate tactile feedback for in-hand manipulation. Results show that by exploiting the dense tactile information we can reconstruct the shape of objects with high accu…
Active Cross-Modal Visuo-Tactile Perception of Deformable Linear Objects
Dipartimento di Ingegneria, Università degli Studi della Campania Luigi Vanvitelli（工程学院，坎帕尼亚路易吉·范维蒂利大学）；Dipartimento di Ingegneria dell’Informazione, Università degli Studi di Padova（信息工程学院，帕多瓦大学）
Adaptive Visuo-Tactile Fusion with Predictive Force Attention for Dexterous Manipulation
Center on Frontiers of Computing Studies, School of Computer Science, Peking University（前沿计算研究中心，计算机科学学院，北京大学）；PKU-Agibot Lab, School of Computer Science, Peking University（PKU-Agibot实验室，计算机科学学院，北京大学）；National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机科学学院，北京大学）；Ant Group（蚂蚁集团）；Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）
Tactile-based Multimodal Fusion in Embodied Intelligence: A Survey of Vision, Language, and Contact-Driven Paradigms
School of Electronic Science and Engineering, Xi’an Jiaotong University, China（西安交通大学电子科学与技术学院）；Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou), China（香港科技大学（广州）人工智能研究所）；State Key Laboratory for Novel Software Technology, Nanjing University, China（南京大学新型软件技术国家重点实验室）；Purple Mountain Laboratory, China（紫金山实验室）；Institute for Math & AI, Wuhan University, China（武汉大学数学与人工智能学院）；Centre for AI and Data Science Innovation and the School of Science and Engineering, James Cook University, Australia（詹姆斯库克大学人工智能与数据科学创新中心及科学与工程学院）；School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China（北京邮电大学人工智能学院）；Institute of Big Data, Fudan University, China（复旦大学大数据研究院）；Linkerbot (Beijing) Technology Co., Ltd, China（北京链动科技有限公司）；School of Engineering, Swinburne University of Technology, Melbourne（斯威本技术大学工程学院）
