arXiv 2507.237342025-07-31cs.CVcs.RO

RAGNet：面向通用抓取的大规模基于推理的可供性分割基准

RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Grasping

The Chinese University of Hong Kong（香港中文大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Dexmal；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；SKL-IOTSC, CIS, University of Macau（澳门科学馆-物联网与智能系统研究中心，澳门大学）

Dongming Wu, Yanping Fu, Saike Huang, Yingfei Liu, Fan Jia, Nian Liu, Feng Dai, Tiancai Wang, Rao Muhammad Anwer, Fahad Shahbaz Khan, Jianbing Shen

备注：: Accepted by ICCV 2025. The code is at https://github.com/wudongming97/AffordanceNet

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有研究缺乏基于推理的大规模可供性预测数据、开放世界效果受限的问题，该研究构建了含27.3万张图像的RAGNet基准，提出AffordanceNet抓取框架，验证了其强大的开放世界泛化能力。

摘要：

通用机器人抓取系统需要在多样化的开放世界场景中，根据人类指令实现准确的物体可供性感知。然而，当前研究存在缺乏基于推理的大规模可供性预测数据的问题，这引发了人们对其开放世界有效性的极大担忧。为解决这一局限，我们构建了一个带有类人指令的大规模面向抓取的可供性分割基准，名为RAGNet。该基准包含27.3万张图像、180个类别以及2.6万条推理指令。图像涵盖了多样化的具身数据领域，例如野外场景、机器人视角、第一人称视角甚至仿真数据。这些图像都经过了可供性图的精细标注，同时我们通过移除指令中的类别名称、仅提供功能描述的方式，大幅提升了语言指令的难度。此外，我们提出了一个综合的基于可供性的抓取框架，名为AffordanceNet，该框架由一个在我们的大规模可供性数据上预训练的VLM（视觉语言模型），以及一个以可供性图为条件来抓取目标的抓取网络组成。在可供性分割基准和真实机器人操作任务上开展的大量实验表明，我们的模型具备强大的开放世界泛化能力。我们的数据和代码可在https://github.com/wudongming97/AffordanceNet获取。

英文摘要：

General robotic grasping systems require accurate object affordance perception in diverse open-world scenarios following human instructions. However, current studies suffer from the problem of lacking reasoning-based large-scale affordance prediction data, leading to considerable concern about open-world effectiveness. To address this limitation, we build a large-scale grasping-oriented affordance segmentation benchmark with human-like instructions, named RAGNet. It contains 273k images, 180 categories, and 26k reasoning instructions. The images cover diverse embodied data domains, such as wild, robot, ego-centric, and even simulation data. They are carefully annotated with an affordance map, while the difficulty of language instructions is largely increased by removing their category name and only providing functional descriptions. Furthermore, we propose a comprehensive affordance-based grasping framework, named AffordanceNet, which consists of a VLM pre-trained on our massive affordance data and a grasping network that conditions an affordance map to grasp the target. Extensive experiments on affordance segmentation benchmarks and real-robot manipulation tasks show that our model has a powerful open-world generalization ability. Our data and code is available at https://github.com/wudongming97/AffordanceNet.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。