RoboDexVLM：面向灵巧机器人操作的视觉语言模型驱动任务规划与运动控制
RoboDexVLM: Visual Language Model-Enabled Task Planning and Motion Control for Dexterous Robot Manipulation
AI总结：
本文提出RoboDexVLM框架，结合VLM任务规划与语言引导的零样本灵巧抓取算法，解决长程复杂操作任务，实现开放词汇的灵巧机器人操作。
摘要：
本文介绍了RoboDexVLM，这是一个为配备灵巧手的协作机械臂量身定制的机器人任务规划和抓取检测的创新框架。以往的方法侧重于简化和有限的操作任务，往往忽略了以长程方式抓取多种物体的复杂性。相比之下，我们提出的框架利用能够抓取不同形状和大小物体的灵巧手，同时根据自然语言命令执行任务。该方法包含以下核心组件：首先，设计了一个具有任务级恢复机制的鲁棒任务规划器，利用视觉语言模型（VLMs），使系统能够解释和执行长序列任务的开放词汇命令。其次，提出了一种基于机器人运动学和形式方法的语言引导灵巧抓取感知算法，专为针对多样物体和命令的零样本灵巧操作而设计。综合实验结果验证了RoboDexVLM在处理长程场景和执行灵巧抓取方面的有效性、适应性和鲁棒性。这些结果突出了该框架在复杂环境中运行的能力，展示了其在开放词汇灵巧操作方面的潜力。我们的开源项目页面可在https://henryhcliu.github.io/robodexvlm找到。
英文摘要：
This paper introduces RoboDexVLM, an innovative framework for robot task planning and grasp detection tailored for a collaborative manipulator equipped with a dexterous hand. Previous methods focus on simplified and limited manipulation tasks, which often neglect the complexities associated with grasping a diverse array of objects in a long-horizon manner. In contrast, our proposed framework utilizes a dexterous hand capable of grasping objects of varying shapes and sizes while executing tasks based on natural language commands. The proposed approach has the following core components: First, a robust task planner with a task-level recovery mechanism that leverages vision-language models (VLMs) is designed, which enables the system to interpret and execute open-vocabulary commands for long sequence tasks. Second, a language-guided dexterous grasp perception algorithm is presented based on robot kinematics and formal methods, tailored for zero-shot dexterous manipulation with diverse objects and commands. Comprehensive experimental results validate the effectiveness, adaptability, and robustness of RoboDexVLM in handling long-horizon scenarios and performing dexterous grasping. These results highlight the framework's ability to operate in complex environments, showcasing its potential for open-vocabulary dexterous manipulation. Our open-source project page can be found at https://henryhcliu.github.io/robodexvlm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
