学习双手多指视觉触觉技能
Learning Visuotactile Skills with Two Multifingered Hands
AI总结：
本研究开发了低成本遥操作系统HATO并改造假手，利用视觉触觉数据学习双臂多指操作技能，完成长时域高精度任务，并探究数据规模、模态和预处理对策略学习的影响。
摘要：
为了复制类人的灵巧性、感知体验和运动模式，我们探索使用具有多指手和视觉触觉数据的双臂系统从人类演示中学习。存在两个重大挑战：缺乏适用于带多指手的双臂设置的经济实惠且可访问的遥操作系统，以及配备触觉传感的多指手硬件稀缺。为了应对第一个挑战，我们开发了HATO，一种低成本的臂手遥操作系统，利用现成的电子元件，并辅以一套软件套件，支持高效的数据收集；该综合软件套件还支持多模态数据处理、可扩展的策略学习和流畅的策略部署。为了应对后一个挑战，我们引入了一种新颖的硬件改造，将两只配备触觉传感器的假手重新用于研究。使用从我们系统收集的视觉触觉数据，我们学习技能以完成长时域、高精度任务，这些任务在没有多指灵巧性和触觉反馈的情况下难以实现。此外，我们实证研究了数据集大小、传感模态和视觉输入预处理对策略学习的影响。我们的结果标志着从视觉触觉数据进行双臂多指操作迈出了有希望的一步。视频、代码和数据集可在https://toruowo.github.io/hato/找到。
英文摘要：
Aiming to replicate human-like dexterity, perceptual experiences, and motion patterns, we explore learning from human demonstrations using a bimanual system with multifingered hands and visuotactile data. Two significant challenges exist: the lack of an affordable and accessible teleoperation system suitable for a dual-arm setup with multifingered hands, and the scarcity of multifingered hand hardware equipped with touch sensing. To tackle the first challenge, we develop HATO, a low-cost hands-arms teleoperation system that leverages off-the-shelf electronics, complemented with a software suite that enables efficient data collection; the comprehensive software suite also supports multimodal data processing, scalable policy learning, and smooth policy deployment. To tackle the latter challenge, we introduce a novel hardware adaptation by repurposing two prosthetic hands equipped with touch sensors for research. Using visuotactile data collected from our system, we learn skills to complete long-horizon, high-precision tasks which are difficult to achieve without multifingered dexterity and touch feedback. Furthermore, we empirically investigate the effects of dataset size, sensing modality, and visual input preprocessing on policy learning. Our results mark a promising step forward in bimanual multifingered manipulation from visuotactile data. Videos, code, and datasets can be found at https://toruowo.github.io/hato/ .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
