TypeTele：通过灵巧操作类型释放遥操作中的灵巧性
TypeTele: Releasing Dexterity in Teleoperation by Dexterous Manipulation Types
- School of Computer Science and Engineering, Sun Yat-sen University, China（中山大学计算机科学与工程学院）
- Stanford University, USA（斯坦福大学）
- Imperial College London, UK（伦敦帝国理工学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
TypeTele提出类型引导的灵巧遥操作系统，通过引入灵巧操作类型库和MLLM辅助类型检索，使灵巧手突破人类运动模式限制，显著提升多样复杂任务的成功率。
AI中文摘要：
灵巧遥操作在机器人操作中对于真实世界数据采集和远程机器人控制起着至关重要的作用。以往的灵巧遥操作大多依赖手部重定向来尽可能模仿人类手部姿态。然而，这些方法可能无法充分利用灵巧手固有的灵巧性，因为灵巧手能够凭借其相对于人手的结构优势执行独特动作。为了解决这一局限，我们提出TypeTele，一个类型引导的灵巧遥操作系统，使灵巧手能够执行不受人类运动模式约束的动作。这是通过将灵巧操作类型引入遥操作系统来实现的，使操作者能够采用合适的类型来完成特定任务。为了支持该系统，我们构建了一个可扩展的灵巧操作类型库，以覆盖操作任务中使用的全面灵巧姿态。在遥操作过程中，我们采用一个MLLM（多模态大语言模型）辅助的类型检索模块，根据具体任务和操作者指令识别最合适的操作类型。大量真实世界遥操作和模仿学习实验表明，引入操作类型能够显著充分发挥灵巧机器人执行多样且复杂任务的能力，并获得更高的成功率。
英文摘要：
Dexterous teleoperation plays a crucial role in robotic manipulation for real-world data collection and remote robot control. Previous dexterous teleoperation mostly relies on hand retargeting to closely mimic human hand postures. However, these approaches may fail to fully leverage the inherent dexterity of dexterous hands, which can execute unique actions through their structural advantages compared to human hands. To address this limitation, we propose TypeTele, a type-guided dexterous teleoperation system, which enables dexterous hands to perform actions that are not constrained by human motion patterns. This is achieved by introducing dexterous manipulation types into the teleoperation system, allowing operators to employ appropriate types to complete specific tasks. To support this system, we build an extensible dexterous manipulation type library to cover comprehensive dexterous postures used in manipulation tasks. During teleoperation, we employ a MLLM (Multi-modality Large Language Model)-assisted type retrieval module to identify the most suitable manipulation type based on the specific task and operator commands. Extensive experiments of real-world teleoperation and imitation learning demonstrate that the incorporation of manipulation types significantly takes full advantage of the dexterous robot's ability to perform diverse and complex tasks with higher success rates.
