Visual-tactile Fusion for Transparent Object Grasping in Complex Backgrounds
Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);RISC-V International Open Source Laboratory(RISC-V国际开源实验室);Tsinghua-Berkeley Shenzhen Institute(清华-伯克利深圳学院);Institute of Artificial Intelligence, Collaborative Innovation Center for the Marine Artificial Intelligence, Shanghai University(上海大学人工智能学院、海洋人工智能协同创新中心)
The accurate detection and grasping of transparent objects are challenging but of significance to robots. Here, a visual-tactile fusion framework for transparent object grasping under complex backgrounds and variant light conditions is proposed, including the grasping position detection, tactile calibration, and visual-tactile fusion based classification. First, a multi-scene synthetic grasping dataset generation method with a Gaussian distribution based data annotation is proposed. Besides, a novel grasping network named TGCNN is proposed for grasping position detection, showing good results in both synthetic and real scenes. In tactile calibration, inspired by human grasping, a fully convolutional network based tactile feature extraction method and a central location based adaptive grasping strategy are designed, improving the success rate by 36.7% compared to direct grasping. Furthermore, a visual-tactile fusion method is proposed for transparent objects classification, which improves the classification accuracy by 34%. The proposed framework synergizes the advantages of vision and touch, and greatly improves the grasping efficiency of transparent objects.
Phy-Tac: Toward Human-Like Grasping via Physics-Conditioned Tactile Goals
Department of Mechanical Engineering, The Hong Kong Polytechnic University (PolyU)(香港理工大学机械工程系);Research Institute for Smart Ageing (RISA), PolyU(PolyU智能老龄化研究所以);Ningbo Institute of Digital Twin, Eastern Institute of Technology (EIT)(宁波数字孪生研究所,东部技术研究所);Department of Mechanical and Energy Engineering, Southern University of Science and Technology(南方科技大学机械与能源工程系)
Transparent Fragments Contour Estimation via Visual-Tactile Fusion for Autonomous Reassembly
School of Advanced Manufacturing, Sun Yat-sen University(中山大学先进制造学院);School of Electronics and Communication Engineering, Sun Yat-sen University(中山大学电子与通信工程学院);School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院)
Learning Robust Grasping Strategy Through Tactile Sensing and Adaption Skill
School of Power and Mechanical Engineering, Wuhan University(武汉大学动力与机械学院);School of Technological Sciences, Wuhan University(武汉大学技术科学学院);School of Microelectronics, Wuhan University(武汉大学微电子学院);School of Computer Science, Wuhan University(武汉大学计算机学院)
Grasping in the Dark: Zero-Shot Object Grasping Using Tactile Feedback
Perception and Robotics Group, University of Maryland, College Park(马里兰大学帕克分校感知与机器人小组);University of Maryland, College Park(马里兰大学帕克分校);University of Maryland(马里兰大学)
TouchDrive: Electronics-Free Tactile Sensing Interface for Assistive Grasping
Division of Microsystems Technology, Department of Electrical Engineering(微系统技术部门);Cyber-Physical Systems Lab, Department of Information Technology(网络物理系统实验室)
TacFinRay: Soft Tactile Fin-Ray Finger with Indirect Tactile Sensing for Robust Grasping
Graduate School of Data Science at Kyungpook National University(Kyungpook National University 数据科学研究生院);University of Bristol(布里斯托大学);Bristol Robotics Laboratory(布里斯托机器人实验室)
Learning Fine Pinch-Grasp Skills using Tactile Sensing from A Few Real-world Demonstrations
School of Informatics, University of Edinburgh(爱丁堡大学信息学院);Touchlab Limited(Touchlab有限公司);Department of Engineering Mathematics, University of Bristol(布里斯托尔大学工程数学系);Department of Computer Science, University College London(伦敦大学学院计算机科学系)
Fusing Visuo-Tactile Perception into Kernelized Synergies for Robust Grasping and Fine Manipulation of Non-rigid Objects
Istituto Italiano di Tecnologia(意大利理工学院);University of Naples Federico II(那不勒斯费德里科二世大学);The Chinese University of Hong Kong(香港中文大学)
使用机器人手处理非刚性物体需要一个框架,该框架不仅包含人类水平的灵活性和认知,而且还包含多感官信息和系统动力学,以实现强大而精细的交互。在这项研究中,我们以前开发的内核化协同框架,灵感来自人类行为上重用相同的子空间进行抓取和操纵,增强了视觉触觉感知,以自主和灵活地适应未知物体。为了检测物体并估计其姿态,利用了简化的视觉管道,使用具有欧几里德聚类和SVM分类器的RANSAC算法。为了调节交互的努力,同时抓住和操纵非刚性物体,触觉反馈使用T40 S shokac芯片传感器,生成3D力信息,被纳入。此外,不同的核函数进行了检查,在内核化的协同框架,以评估其性能和潜力,对任务的再现性,执行,泛化和协同可重用性。通过机器人手-臂系统的实验验证了改进后的框架在非刚性物体稳定抓取和灵巧操作方面的能力和实用性。
Grasping Deformable Objects via Reinforcement Learning with Cross-Modal Attention to Visuo-Tactile Inputs
Dept. of Electronic Engineering at Sogang University(ソガン大学电子工程系);Dept. of Immersive Media and Engineering at Sungkyunkwan University(顺天大学沉浸媒体与工程系);College of Medicine, Yonsei University(延世大学医学院)
Visual-Tactile Sensing for Real-time Liquid Volume Estimation in Grasping
Xi’an-Jiaotong Liverpool University(西交利物浦大学);The University of Hong Kong(香港大学);City University of Hong Kong(香港城市大学);Southern University of Science and Technology(南方科技大学)
Tactile Model O: Fabrication and testing of a 3d-printed, three-fingered tactile robot hand
University of Bristol(布里斯托大学);Bristol Robotics Laboratory(布里斯托机器人实验室)
为机器人手带来触觉将允许更有效的抓握,以及类似人类触摸的广泛好处。在这里,我们提出了一个3D打印的三指触觉机器人手,包括一个定制的OpenHand Model O,用于在每个手指的远端指骨中容纳TacTip柔软的仿生触觉传感器。我们期望将这种欠驱动手的抓取能力与复杂的触觉感知相结合,将导致一个有效的机器人手研究平台-触觉模型O(T-MO)。该设计使用了三个JeVois机器视觉系统,每个系统都包括一个位于触觉指尖的微型摄像头和一个位于手底部的处理模块。为了评估T-MO的能力,我们使用YCB对象集上的抓取器评估基准对其抓取性能进行基准测试。触觉感知的能力进行评估,通过执行触觉对象分类的26个对象,并预测是否把握将成功地解除每个对象。结果与最新技术水平一致,利用了应用于触觉图像输出的深度学习的进步。总的来说,这项工作表明,T-MO是一个有效的平台,机器人手的研究,我们希望它打开了一系列的应用在自主物体处理。补充视频:https://youtu.be/RTcCpgffCrQ。