UniAff：面向工具使用与关节体操作的视觉-语言模型可供性统一表示
UniAff: A Unified Representation of Affordances for Tool Usage and Articulation with Vision-Language Models
AI总结：
本文提出UniAff统一范式，通过构建包含900个关节体和600个工具的数据集，并利用多模态大语言模型推断物体中心的三维运动约束与可供性，显著提升了机器人对工具和关节体操作的泛化能力。
摘要：
以往关于机器人操作的研究建立在对底层三维运动约束和可供性的有限理解之上。为应对这些挑战，我们提出了一种全面的范式，称为UniAff，它将以三维物体为中心的操作和任务理解整合到一个统一的表述中。具体而言，我们构建了一个标注了操作相关关键属性的数据集，包含来自19个类别的900个关节体和来自12个类别的600个工具。此外，我们利用多模态大语言模型（MLLMs）来推断以物体为中心的操作任务表示，包括可供性识别和关于三维运动约束的推理。在仿真和真实环境中的综合实验表明，UniAff显著提高了机器人对工具和关节体操作的泛化能力。我们希望UniAff未来能够作为统一机器人操作任务的通用基线。图像、视频、数据集和代码已发布在项目网站上：https://sites.google.com/view/uni-aff/home
英文摘要：
Previous studies on robotic manipulation are based on a limited understanding of the underlying 3D motion constraints and affordances. To address these challenges, we propose a comprehensive paradigm, termed UniAff, that integrates 3D object-centric manipulation and task understanding in a unified formulation. Specifically, we constructed a dataset labeled with manipulation-related key attributes, comprising 900 articulated objects from 19 categories and 600 tools from 12 categories. Furthermore, we leverage MLLMs to infer object-centric representations for manipulation tasks, including affordance recognition and reasoning about 3D motion constraints. Comprehensive experiments in both simulation and real-world settings indicate that UniAff significantly improves the generalization of robotic manipulation for tools and articulated objects. We hope that UniAff will serve as a general baseline for unified robotic manipulation tasks in the future. Images, videos, dataset, and code are published on the project website at:https://sites.google.com/view/uni-aff/home
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
