arXiv 2610.203202026-10-15cs.RO

TacClaw-Infinity：用于灵巧操作的大规模视觉-触觉-语言-动作数据集

TacClaw-Infinity: A Large-Scale Vision-Tactile-Language-Action Dataset for Dexterous Manipulation

Fucai Zhu, Xidong Zhang, Yichi Zhang, Yipan Wang, Fubo Wang, Zibin Wu, Kaiqi Li, Jianhao Jiao, Shihan Lu, Di Hu, Mengdi Xu, Siyu Zhu, Xiaoxiao Long, Yao Yao, Xu… 展开作者

Fucai Zhu, Xidong Zhang, Yichi Zhang, Yipan Wang, Fubo Wang, Zibin Wu, Kaiqi Li, Jianhao Jiao, Shihan Lu, Di Hu, Mengdi Xu, Siyu Zhu, Xiaoxiao Long, Yao Yao, Xun Cao, Zhuo Xu, Koichi Hashimoto, Qifeng Chen, Zhe Li, Jianfei Yang, Haofei Lu, Danica Kragic, Shuicheng Yan, Jianghua Duan, Michael Yu Wang, Weihao Yuan

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

该研究提出TacClaw-Infinity大规模VTLA数据集，通过全球分布式收集构建，开发了多模态处理流程，在数据和模型层面验证其有效性，实现跨异构机器人的无特定演示迁移。

摘要：

大规模机器人学习越来越依赖多样化的人类演示，但触觉操作数据难以扩展，现有多模态数据集往往缺乏一致的质量控制和结构化监督。我们提出TacClaw-Infinity，这是一个用于接触丰富操作的大规模结构化视觉-触觉-语言-动作（VTLA）数据集，通过TacClaw收集，TacClaw是一种支持触觉的UMI风格手持界面，可记录同步的视觉观测、指尖触觉信号和末端执行器轨迹，且数据收集期间无需目标机器人。TacClaw-Infinity通过在家庭、实验室、工业工作场所、零售店及户外环境的全球分布式收集构建，包含学术和工业合作者贡献的任务。这些工作共同产出了超过10000小时、100万集、10亿帧，涵盖1000多种任务类型。我们开发了可扩展的轨迹重建、多模态质量控制、具身感知过滤和结构化标注流程，涵盖以接触为中心的分割、子任务描述和空间定位。我们在数据和模型层面验证TacClaw-Infinity：数据层面，我们评估收集界面、整理后演示的动作空间兼容性及结构化标注；模型层面，我们在三种异构机器人上展示跨具身迁移，无需特定于机器人的演示。

英文摘要：

Large-scale robot learning increasingly relies on diverse human demonstrations, yet tactile manipulation data remain difficult to scale, and existing multimodal datasets often lack consistent quality control and structured supervision. We present TacClaw-Infinity, a large-scale structured Vision-Tactile-Language-Action (VTLA) dataset for contact-rich manipulation, collected with TacClaw, a tactile-enabled UMI-style handheld interface that records synchronized visual observations, fingertip tactile signals, and end-effector trajectories without requiring a target robot during data collection. TacClaw-Infinity is built through a distributed worldwide collection across homes, laboratories, industrial workplaces, retail stores, and outdoor environments including tasks contributed by academic and industrial collaborators. Together, these efforts yield over 10,000 hours, 1 million episodes, and 1 billion frames spanning more than 1,000 task types. We develop a scalable pipeline for trajectory reconstruction, multimodal quality control, embodiment-aware filtering, and structured annotation, covering contact-centric segmentation, subtask description, and spatial grounding. We validate TacClaw-Infinity at both the data and model levels: At the data level, we assess the collection interface, the action-space compatibility of the curated demonstrations, and the structured annotations, while for the model level, we demonstrate cross-embodiment transfer across three heterogeneous robots without robot-specific demonstrations.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。