Human-robot co-manipulation of soft materials, such as fabrics, composites, and sheets of paper/cardboard, is a challenging operation that presents several relevant industrial applications. Estimating the deformation state of the co-manipulated material is one of the main challenges. Viable methods provide the indirect measure by calculating the human-robot relative distance. In this paper, we develop a data-driven model to estimate the deformation state of the material from a depth image through a Convolutional Neural Network (CNN). First, we define the deformation state of the material as the relative roto-translation from the current robot pose and a human grasping position. The model estimates the current deformation state through a Convolutional Neural Network, specifically a DenseNet-121 pretrained on ImageNet.The delta between the current and the desired deformation state is fed to the robot controller that outputs twist commands. The paper describes the developed approach to acquire, preprocess the dataset and train the model. The model is compared with the current state-of-the-art method based on a skeletal tracker from cameras. Results show that our approach achieves better performances and avoids the various drawbacks caused by using a skeletal tracker.Finally, we also studied the model performance according to different architectures and dataset dimensions to minimize the time required for dataset acquisition
EgoPhys: Estimating Peak Contact Force and Mechanical Work from Egocentric Manipulation Video
School of Artificial Intelligence, Shandong University(山东大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Mechanical Engineering, Tianjin University(天津大学机械工程学院)
SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-body Manipulation
Fudan University, China(复旦大学);Shanghai Artificial Intelligence Laboratory, China(上海人工智能实验室);Shanghai Jiao Tong University, China(上海交通大学);The Chinese University of Hong Kong, China(香港中文大学);The University of Hong Kong, China(香港大学)
SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects (Early Version)
Tuojing Intelligence(拓景智能);Tsinghua University(清华大学);King’s College London(伦敦国王学院);Southeast University(东南大学);Stevens Institute of Technology(史蒂文斯理工学院);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));University of Manchester(曼彻斯特大学);Simple AI(简单人工智能公司);Imperial College London(伦敦帝国学院);Carnegie Mellon University(卡内基梅隆大学);Zhejiang University(浙江大学);Beihang University(北京航空航天大学);The University of Hong Kong(香港大学)
Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration
University of California at Berkeley(加州大学伯克利分校);Technische Universität Berlin(柏林工业大学)
灵巧的多指手可以完成精细的操作行为,这是不可行的简单的机器人抓手。然而,复杂的多指手往往是昂贵和脆弱的。低成本的软手为更传统的设备提供了一种有吸引力的替代方案,但在传感和致动方面存在相当大的挑战,使得它们难以应用于更复杂的操作任务。在本文中,我们描述了一种方法,从示范,可用于训练软机器人手执行灵巧的操作任务的学习。我们的方法使用以对象为中心的演示,其中人类用自己的手演示所需的操作对象的运动,机器人自主学习模仿这些演示使用强化学习。我们提出了一种新的算法,使我们能够混合和选择一个子集的最可行的示范学习模仿的硬件,我们使用的指导政策搜索框架的扩展,使用多个示范学习概括的神经网络政策。我们在RBO Hand 2上展示了我们的方法,学习了转动阀门,操纵算盘和抓握的运动技能。
DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics
University of California, San Diego(加州大学圣地亚哥分校);Tsinghua University(清华大学);Autodesk(欧特克);UMass Amherst(马萨诸塞大学阿默斯特分校);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室)
AeRSoM: An Aerial Rigid-Soft Integrated Manipulator for Contact-Rich Manipulation
Hunan University(湖南大学);National Engineering Research Center for Robot Visual Perception and Control Technology(国家机器人视觉感知与控制技术工程研究中心);Beihang University(北京航空航天大学)
Dexterous Soft Hands Linearize Feedback-Control for In-Hand Manipulation
Robotics and Biology Laboratory, Technische Universität Berlin(柏林工业大学机器人与生物实验室);Science of Intelligence, Research Cluster of Excellence, Berlin(柏林卓越研究集群智能科学)
NoContactNoWorries: Estimating Contact through Vision and Proprioception for In-Hand Dexterous Manipulation
Robotics Research Center (RRC), International Institute of Information Technology (IIIT), Hyderabad, India(印度海得拉巴国际信息技术研究所机器人研究中心);Department of Computer Science, The University of Manchester(曼彻斯特大学计算机科学系)
Intersection-free Robot Manipulation with Soft-Rigid Coupled Incremental Potential Contact
ZenusTech Inc.(泽努斯科技公司);Shanghai Jiao Tong University(上海交通大学);SJTU-Yale Joint Center of Biostatistics and Data Science, National Center for Translational Medicine, Shanghai Jiao Tong University(上海交通大学-耶鲁大学生物统计与数据科学联合中心,上海交通大学国家转化医学中心)
Surface-based Manipulation Using Tunable Compliant Porous-Elastic Soft Sensing
Soft BioRobotics Perception Group at Istituto Italiano di Tecnologia(意大利技术研究院软生物机器人感知组);University of Genoa(热那亚大学);Biorobotics Institute, Scuola Superiore Sant’Anna(圣安娜高等学院生物机器人研究所);Soft Robotics Lab, National University of Singapore(新加坡国立大学软机器人实验室)