arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.14550cs.RO
收藏

TacUMI: 一种多模态通用操控接口用于接触密集型任务

TacUMI: A Multi-Modal Universal Manipulation Interface for Contact-Rich Tasks

School of Computation, Information and Technology, Technical University of Munich(技术大学慕尼黑计算、信息与技术学院);Agile Robots SE(敏捷机器人公司);State Key Laboratory for Novel Software Technology and the School of Science and Technology, Nanjing University (Suzhou Campus)(南京大学软件新技术国家重点实验室及科学与技术学院(苏州校区));Shanghai University(上海大学)

Tailai Cheng, Kejia Chen, Lingyun Chen, Liding Zhang, Yue Zhang, Yao Ling, Mahdi Hamad, Zhenshan Bing, Fan Wu, Karan Sharma, Alois Knoll

AI总结:

TacUMI通过整合多种传感器,提供了一种多模态数据采集系统,用于提高接触密集型任务中多模态演示的分割和收集效率。

摘要:

任务分解对于理解和学习复杂长周期操控任务至关重要。特别是对于涉及丰富物理交互的任务,仅依靠视觉观察和机器人本体信息往往无法揭示底层事件转换。这提出了对高效收集高质量多模态数据以及鲁棒分割方法的需求,以将演示分解为有意义的模块。基于手持演示设备通用操控接口(UMI)的概念,我们引入了TacUMI,一种多模态数据采集系统,其整合了ViTac传感器、力矩传感器和位姿追踪器,设计成紧凑且兼容机器人的机械手,能够在人类演示过程中同步采集所有这些模态。我们随后提出了一种多模态分割框架,利用时间模型来检测序列操控中的语义有意义的事件边界。在具有挑战性的电缆安装任务上的评估显示,分割准确率超过90%,并且在增加更多模态的情况下实现了显著提升,这验证了TacUMI在接触密集型任务中多模态演示的可扩展收集和分割方面建立了实用基础。

英文摘要:

Task decomposition is critical for understanding and learning complex long-horizon manipulation tasks. Especially for tasks involving rich physical interactions, relying solely on visual observations and robot proprioceptive information often fails to reveal the underlying event transitions. This raises the requirement for efficient collection of high-quality multi-modal data as well as robust segmentation method to decompose demonstrations into meaningful modules. Building on the idea of the handheld demonstration device Universal Manipulation Interface (UMI), we introduce TacUMI, a multi-modal data collection system that integrates additionally ViTac sensors, force-torque sensor, and pose tracker into a compact, robot-compatible gripper design, which enables synchronized acquisition of all these modalities during human demonstrations. We then propose a multi-modal segmentation framework that leverages temporal models to detect semantically meaningful event boundaries in sequential manipulations. Evaluation on a challenging cable mounting task shows more than 90 percent segmentation accuracy and highlights a remarkable improvement with more modalities, which validates that TacUMI establishes a practical foundation for both scalable collection and segmentation of multi-modal demonstrations in contact-rich tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑