arXiv 2607.010672026-07-02cs.ROcs.CV

面向灵巧机器人操作的人为中心的可迁移触觉预训练

Human-Centric Transferable Tactile Pre-Training for Dexterous Robotic Manipulation

Peking University（北京大学）；BeingBeyond；Tsinghua University（清华大学）

Chi Zhang, Penglin Cai, Ziheng Xi, Haoqi Yuan, Hao Luo, Wanpeng Zhang, Sipeng Zheng, Chaoyi Xu, Zongqing Lu

备注：: The first two authors contribute equally. Orders are decided by flipping a coin

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出H-Tac大规模触觉-动作数据集和可迁移触觉预训练（TTP）系统，通过统一触觉与动作空间弥合人机差距，利用触觉专家预测未来触觉以建模接触动力学，显著提升灵巧操作的泛化与精细控制能力。

摘要：

作为灵巧和接触密集型任务的关键模态，触觉感知提供了无法从视觉可靠推断的精确力反馈。然而，受限于硬件和数据采集系统，现有的触觉数据集规模小且接触覆盖范围窄。同时，具有触觉模态的视觉-语言-动作（VLA）模型受限于动力学无关的后训练，限制了在下游任务上的性能上限。在本文中，我们提出了H-Tac，一个大规模触觉-动作数据集，包含160小时的自我中心人类视频，涵盖超过300个任务和13.5万个片段。在此基础上，我们提出了可迁移触觉预训练（TTP），一个基于人类数据进行触觉预训练的系统，用于精细的机器人任务。为了弥合人类与机器人之间的差距，我们在预训练和后训练阶段使用统一的触觉和动作空间，在从人到机器人的迁移过程中保留先验知识。通过利用触觉专家进行未来触觉预测，我们的框架显式地建模了接触动力学和精确的物理交互。在仿真和真实机器人上的大量实验表明，我们的模型实现了卓越的性能，展现出鲁棒的泛化和精细操作能力。TTP通过从人到机器人的迁移为可扩展的触觉预训练铺平了道路。

英文摘要：

As an essential modality for dexterous and contact-rich tasks, tactile sensing provides precise force feedback that cannot be reliably inferred from vision. However, limited by hardware and data collection systems, existing datasets with tactility remain small in scale and narrow in contact coverage. Meanwhile, Vision-Language-Action (VLA) models with tactile modality are constrained on dynamics-agnostic post-training, which limits the performance ceiling on downstream tasks. In this paper, we present H-Tac, a large-scale tactile-action dataset with 160-hour egocentric human videos containing more than 300 tasks and 135k episodes. Building upon this, we propose Transferable Tactile Pre-Training (TTP), a system of tactile-based pre-training on human data for fine-grained robotic tasks. To bridge the gap between humans and robots, we use unified tactile and action spaces throughout the pre-training and post-training phases, preserving prior knowledge during human-to-robot transfer. By leveraging a tactile expert for future tactile prediction, our framework explicitly models the contact dynamics and precise physical interactions. Extensive experiments in simulation and on real robots demonstrate that our model achieves superior performance, exhibiting robust generalization and fine-grained manipulation capabilities. TTP paves the way for scalable tactile pre-training via human-to-robot transfer.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。