arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.11941cs.RO
收藏

GauTOAO：基于高斯的目标导向物体可操作性

GauTOAO: Gaussian-based Task-Oriented Affordance of Objects

School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）

Jiawen Wang, Dingsheng Luo

备注：
6 pages
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

GauTOAO利用高斯和视觉-语言模型，零样本预测物体任务导向可操作性区域，通过DINO特征提取3D掩码，实现更准确TOAO提取，提升机器人任务性能。

摘要：

当你的机器人使用灵巧手或夹爪抓取物体时，它应该理解物体的任务导向可操作性（TOAO），因为不同的任务往往需要关注物体的特定部分。为了解决这一挑战，我们提出了GauTOAO，一个基于高斯的目标导向物体可操作性框架，它利用视觉-语言模型以零样本方式，在给定自然语言查询时预测物体与可操作性相关的区域。我们的方法引入了一种新范式：“静态相机，移动物体”，使机器人能够在操作过程中更好地观察和理解手中的物体。GauTOAO通过使用DINO特征提取全面的3D物体掩码，解决了现有方法常常缺乏有效空间分组的问题。然后，该掩码用于条件查询高斯，为指定任务生成物体上的精细语义分布。这种方法实现了更准确的TOAO提取，增强了机器人对物体的理解并提高了任务性能。我们通过真实世界实验验证了GauTOAO的有效性，展示了其跨各种任务的泛化能力。

英文摘要：

When your robot grasps an object using dexterous hands or grippers, it should understand the Task-Oriented Affordances of the Object(TOAO), as different tasks often require attention to specific parts of the object. To address this challenge, we propose GauTOAO, a Gaussian-based framework for Task-Oriented Affordance of Objects, which leverages vision-language models in a zero-shot manner to predict affordance-relevant regions of an object, given a natural language query. Our approach introduces a new paradigm: "static camera, moving object," allowing the robot to better observe and understand the object in hand during manipulation. GauTOAO addresses the limitations of existing methods, which often lack effective spatial grouping, by extracting a comprehensive 3D object mask using DINO features. This mask is then used to conditionally query gaussians, producing a refined semantic distribution over the object for the specified task. This approach results in more accurate TOAO extraction, enhancing the robot's understanding of the object and improving task performance. We validate the effectiveness of GauTOAO through real-world experiments, demonstrating its capability to generalize across various tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑