GLOVER：面向任务导向抓取的泛化开放词汇可供性推理
GLOVER: Generalizable Open-Vocabulary Affordance Reasoning for Task-Oriented Grasping
AI总结：
提出GLOVER框架，通过微调LLM在RGB特征空间预测视觉可供性，并设计非参数抓取规划器AGE，实现了比SOTA快数十倍的开放词汇任务导向抓取。
摘要：
基于人类指令推断任意物体的可供性（即可抓取）部位，对于迈向开放词汇操作的机器人至关重要。然而，当前的抓取规划器受限于有限的视觉语言理解能力和耗时的3D辐射场建模，阻碍了与物体的实时、开放词汇交互。为解决这些局限，我们提出了GLOVER，一个统一的泛化开放词汇可供性推理框架，该框架微调大型语言模型（LLMs），以在RGB特征空间内预测可抓取物体部位的视觉可供性。我们编制了一个包含超过10,000张人机交互图像的数据集，并标注了统一的视觉和语言可供性标签，以实现多模态微调。GLOVER继承了LLMs的世界知识和常识推理能力，促进了更细粒度的物体理解和复杂的工具使用推理。为实现有效的真实世界部署，我们提出了可供性感知抓取估计（AGE），这是一种非参数抓取规划器，可将夹爪姿态与从可供性数据导出的超二次曲面对齐。在30个桌面真实世界场景的评估中，GLOVER在部位识别上达到86.0%的成功率，在抓取上达到76.3%的成功率，其可供性推理速度比此前的最优方法快约29倍，抓取姿态估计速度快约40倍。我们还验证了在不同具身间的泛化能力，展示了其在具有灵巧手的人形机器人上的有效性。
英文摘要：
Inferring affordable (i.e., graspable) parts of arbitrary objects based on human specifications is essential for robots advancing toward open-vocabulary manipulation. Current grasp planners, however, are hindered by limited vision-language comprehension and time-consuming 3D radiance modeling, restricting real-time, open-vocabulary interactions with objects. To address these limitations, we propose GLOVER, a unified Generalizable Open-Vocabulary Affordance Reasoning framework, which fine-tunes the Large Language Models (LLMs) to predict the visual affordance of graspable object parts within RGB feature space. We compile a dataset of over 10,000 images from human-object interactions, annotated with unified visual and linguistic affordance labels, to enable multi-modal fine-tuning. GLOVER inherits world knowledge and common-sense reasoning from LLMs, facilitating more fine-grained object understanding and sophisticated tool-use reasoning. To enable effective real-world deployment, we present Affordance-Aware Grasping Estimation (AGE), a non-parametric grasp planner that aligns the gripper pose with a superquadric surface derived from affordance data. In evaluations across 30 table-top real-world scenes, GLOVER achieves success rates of 86.0% in part identification and 76.3% in grasping, with speeds approximately 29 times faster in affordance reasoning and 40 times faster in grasping pose estimation than the previous state-of-the-art. We also validate the generalization across embodiments, showing effectiveness in humanoid robots with dexterous hands.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
