从人-物交互中学习粒度感知的可供性以实现基于工具的功能性灵巧抓取
Learning Granularity-Aware Affordances from Human-Object Interaction for Tool-Based Functional Dexterous Grasping
AI总结：
本文提出GAAF-Dex框架，通过弱监督学习从人-物交互图像中提取粒度感知的可供性特征，实现功能性区域定位与灵巧手势预测，并构建FAH数据集验证其优于现有方法。
摘要：
为了使机器人能够使用工具，第一步是教会机器人运用灵巧手势精确接触执行任务所需的特定区域。物体的可供性特征在智能体与物体之间的功能性交互中起到桥梁作用。然而，利用这些可供性线索帮助机器人实现功能性工具抓取的问题仍未解决。为此，我们提出了一种粒度感知的可供性特征提取方法，用于定位功能性可供性区域并预测灵巧的粗略手势。我们研究了人类使用工具的内在机制。一方面，我们利用物体-功能手指接触区域的细粒度可供性特征来定位功能性可供性区域。另一方面，我们利用手-物交互区域中高度激活的粗粒度可供性特征来预测抓取手势。此外，我们引入了一个基于模型的后处理模块，将可供性定位和手势预测转化为可执行的机器人动作。由此形成了GAAF-Dex，这是一个完整的框架，它从人-物交互中学习粒度感知的可供性，以实现基于工具的灵巧手功能性抓取。与需要大量数据标注的全监督方法不同，我们采用弱监督方法，从手-物交互的第三人称（Exo）图像中提取相关线索，以监督第一人称（Ego）图像中的特征提取。为支持这一方法，我们构建了一个小规模数据集，即功能性可供性手-物交互数据集（FAH），其中包含近6K张功能性手-物交互的Exo图像和Ego图像。在该数据集上的大量实验表明，我们的方法优于当前最先进的方法。源代码和所建立的数据集可在 https://github.com/yangfan293/GAAF-DEX 获取。
英文摘要：
To enable robots to use tools, the initial step is teaching robots to employ dexterous gestures for touching specific areas precisely where tasks are performed. Affordance features of objects serve as a bridge in the functional interaction between agents and objects. However, leveraging these affordance cues to help robots achieve functional tool grasping remains unresolved. To address this, we propose a granularity-aware affordance feature extraction method for locating functional affordance areas and predicting dexterous coarse gestures. We study the intrinsic mechanisms of human tool use. On one hand, we use fine-grained affordance features of object-functional finger contact areas to locate functional affordance regions. On the other hand, we use highly activated coarse-grained affordance features in hand-object interaction regions to predict grasp gestures. Additionally, we introduce a model-based post-processing module that transforms affordance localization and gesture prediction into executable robotic actions. This forms GAAF-Dex, a complete framework that learns Granularity-Aware Affordances from human-object interaction to enable tool-based functional grasping with dexterous hands. Unlike fully-supervised methods that require extensive data annotation, we employ a weakly supervised approach to extract relevant cues from exocentric (Exo) images of hand-object interactions to supervise feature extraction in egocentric (Ego) images. To support this approach, we have constructed a small-scale dataset, Functional Affordance Hand-object Interaction Dataset (FAH), which includes nearly 6K images of functional hand-object interaction Exo images and Ego images. Extensive experiments on the dataset demonstrate that our method outperforms state-of-the-art methods. The source code and the established dataset are available at https://github.com/yangfan293/GAAF-DEX.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
