arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.09489cs.CV
收藏

语言驱动的抓取检测

Language-driven Grasp Detection

FPT Software AI Center（FPT软件人工智能中心）；Vietnam Automation & Control Institute（越南自动化与控制研究所）；TU Wien（维也纳技术大学）；Imperial College London（伦敦帝国学院）；Ton Duc Thang University（同塔大学）；University of Liverpool（利物浦大学）

An Dinh Vuong, Minh Nhat Vu, Baoru Huang, Nghia Nguyen, Hieu Le, Thieu Vo, Anh Nguyen

备注：
19 pages. Accepted to CVPR24
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出语言驱动的抓取检测数据集Grasp-Anything++及基于扩散模型和对比训练目标的检测方法，实验证明其性能优于现有方法并支持零样本抓取。

摘要：

抓取检测是一个长期存在且复杂的挑战，具有多种工业应用。近年来，许多方法和数据集被提出以解决抓取检测问题。然而，其中大多数并未考虑使用自然语言作为检测抓取位姿的条件。在本文中，我们介绍了Grasp-Anything++，这是一个新的语言驱动的抓取检测数据集，包含100万个样本、超过300万个物体以及超过1000万条抓取指令。我们利用基础模型创建了一个大规模场景语料库，并配有相应的图像和抓取提示。我们将语言驱动的抓取检测任务视为一个条件生成问题。鉴于扩散模型在生成任务中的成功以及语言在该任务中的重要作用，我们提出了一种基于扩散模型的新的语言驱动的抓取检测方法。我们的关键贡献是对比训练目标，该目标明确地有助于去噪过程，以根据语言指令检测抓取位姿。我们说明了我们的方法在理论上具有支持性。大量实验表明，我们的方法优于最先进的方法，并支持真实世界的机器人抓取。最后，我们证明了我们的大规模数据集能够实现零样本抓取检测，并为未来工作提供了一个具有挑战性的基准。项目网站：https://airvlab.github.io/grasp-anything/

英文摘要：

Grasp detection is a persistent and intricate challenge with various industrial applications. Recently, many methods and datasets have been proposed to tackle the grasp detection problem. However, most of them do not consider using natural language as a condition to detect the grasp poses. In this paper, we introduce Grasp-Anything++, a new language-driven grasp detection dataset featuring 1M samples, over 3M objects, and upwards of 10M grasping instructions. We utilize foundation models to create a large-scale scene corpus with corresponding images and grasp prompts. We approach the language-driven grasp detection task as a conditional generation problem. Drawing on the success of diffusion models in generative tasks and given that language plays a vital role in this task, we propose a new language-driven grasp detection method based on diffusion models. Our key contribution is the contrastive training objective, which explicitly contributes to the denoising process to detect the grasp pose given the language instructions. We illustrate that our approach is theoretically supportive. The intensive experiments show that our method outperforms state-of-the-art approaches and allows real-world robotic grasping. Finally, we demonstrate our large-scale dataset enables zero-short grasp detection and is a challenging benchmark for future work. Project website: https://airvlab.github.io/grasp-anything/

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑