像人类一样抓取：从人类本体感觉运动整合中学习可泛化的多指抓取
Grasp Like Humans: Learning Generalizable Multi-Fingered Grasping from Human Proprioceptive Sensorimotor Integration
AI总结：
本文提出一种基于手套的触觉-运动学感知-预测框架，通过模仿学习将人类抓取技能迁移至机器人手，利用图结构和时空图网络实现泛化抓取。
摘要：
触觉和动觉感知对于人类灵巧操作至关重要，通过本体感觉运动整合能够可靠地抓取物体。对于机器人手而言，尽管获取这种触觉和动觉反馈是可行的，但建立从这种感官反馈到运动动作的直接映射仍然具有挑战性。在本文中，我们提出了一种新颖的基于手套的触觉-运动学感知-预测框架，用于通过模仿学习将抓取技能从人类直观自然操作迁移到机器人执行，并通过包括可变形物体在内的泛化抓取任务验证了其有效性。首先，我们集成了一款数据手套，以在关节层面捕获触觉和运动学数据。该手套适用于人类和机器人手，允许在不同场景下从自然的人类手部演示中收集数据。它确保了原始数据格式的一致性，从而能够评估人类和机器人手的抓取性能。其次，我们建立了基于极坐标图结构的统一多模态输入表示。我们明确地将形态差异整合到所设计的表示中，增强了不同演示者和机器人手之间的兼容性。此外，我们引入了触觉-运动学时空图网络（TK-STGN），该网络利用多维子图卷积和基于注意力的LSTM层从图输入中提取时空特征，以预测每个手关节的基于节点的状态。这些预测随后通过力-位置混合映射转换为最终命令。
英文摘要：
Tactile and kinesthetic perceptions are crucial for human dexterous manipulation, enabling reliable grasping of objects via proprioceptive sensorimotor integration. For robotic hands, even though acquiring such tactile and kinesthetic feedback is feasible, establishing a direct mapping from this sensory feedback to motor actions remains challenging. In this paper, we propose a novel glove-mediated tactile-kinematic perception-prediction framework for grasp skill transfer from human intuitive and natural operation to robotic execution based on imitation learning, and its effectiveness is validated through generalized grasping tasks, including those involving deformable objects. Firstly, we integrate a data glove to capture tactile and kinesthetic data at the joint level. The glove is adaptable for both human and robotic hands, allowing data collection from natural human hand demonstrations across different scenarios. It ensures consistency in the raw data format, enabling evaluation of grasping for both human and robotic hands. Secondly, we establish a unified representation of multi-modal inputs based on graph structures with polar coordinates. We explicitly integrate the morphological differences into the designed representation, enhancing the compatibility across different demonstrators and robotic hands. Furthermore, we introduce the Tactile-Kinesthetic Spatio-Temporal Graph Networks (TK-STGN), which leverage multidimensional subgraph convolutions and attention-based LSTM layers to extract spatio-temporal features from graph inputs to predict node-based states for each hand joint. These predictions are then mapped to final commands through a force-position hybrid mapping.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
