DexViTac：收集人类视觉-触觉-运动示范以实现富接触的灵巧操作
DexViTac: Collecting Human Visuo-Tactile-Kinematic Demonstrations for Contact-Rich Dexterous Manipulation
AI总结：
DexViTac通过高保真采集第一人称视觉、高密度触觉感知、末端执行器姿态和手部运动学，构建了超过2400个视觉-触觉-运动学示范的多模态数据集，提升了接触丰富灵巧操作的学习效率和成功率。
摘要：
大规模、高质量的多模态示范对于机器人学习接触丰富的灵巧操作至关重要。尽管以人为中心的数据收集系统降低了扩展的门槛，但在物理交互中捕捉触觉信息却存在困难。为此，我们提出了DexViTac，一种便携式、以人为中心的数据收集系统，专门用于接触丰富的灵巧操作。该系统能够在非结构化、真实环境中高保真地采集第一人称视觉、高密度触觉感知、末端执行器姿态和手部运动学。基于此硬件，我们提出了一种基于运动学的触觉表征学习算法，有效解决了触觉信号中的语义歧义。利用DexViTac的效率，我们构建了一个包含超过2,400个视觉-触觉-运动学示范的多模态数据集。实验表明，DexViTac每小时可收集超过248个示范，并在复杂视觉遮挡下保持稳健。现实部署证实，使用所提出的数据集和学习策略训练的策略在四个具有挑战性的任务中实现了平均成功率超过85%。这种性能显著优于基线方法，从而验证了该系统对学习接触丰富的灵巧操作的实质性改进。项目页面：https://xitong-c.github.io/DexViTac/.
英文摘要：
Large-scale, high-quality multimodal demonstrations are essential for robot learning of contact-rich dexterous manipulation. While human-centric data collection systems lower the barrier to scaling, they struggle to capture the tactile information during physical interactions. Motivated by this, we present DexViTac, a portable, human-centric data collection system tailored for contact-rich dexterous manipulation. The system enables the high-fidelity acquisition of first-person vision, high-density tactile sensing, end-effector poses, and hand kinematics within unstructured, in-the-wild environments. Building upon this hardware, we propose a kinematics-grounded tactile representation learning algorithm that effectively resolves semantic ambiguities within tactile signals. Leveraging the efficiency of DexViTac, we construct a multimodal dataset comprising over 2,400 visuo-tactile-kinematic demonstrations. Experiments demonstrate that DexViTac achieves a collection efficiency exceeding 248 demonstrations per hour and remains robust against complex visual occlusions. Real-world deployment confirms that policies trained with the proposed dataset and learning strategy achieve an average success rate exceeding 85% across four challenging tasks. This performance significantly outperforms baseline methods, thereby validating the substantial improvement the system provides for learning contact-rich dexterous manipulation. Project page: https://xitong-c.github.io/DexViTac/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
