面向灵巧视触觉策略学习的3D触觉规范表示与基于力的预训练
Canonical Representation and Force-Based Pretraining of 3D Tactile for Dexterous Visuo-Tactile Policy Learning
AI总结：
针对3D触觉数据维度高、缺乏预训练模型的问题，提出规范表示方法和基于力的自监督预训练任务，在真实灵巧操作任务上取得78%的平均成功率，验证了方法的有效性。
摘要：
触觉感知在使机器人执行细粒度、高接触任务中发挥着关键作用。然而，由于灵巧手上的触觉传感器覆盖范围大，触觉数据维度极高，这给有效的触觉特征学习带来了巨大挑战，尤其是3D触觉数据，目前既没有大型标准化数据集，也没有强大的预训练骨干网络。为应对这些挑战，我们提出了一种新颖的规范表示方法，降低了3D触觉特征学习的难度，并进一步引入了一项基于力的自监督预训练任务，以捕捉对灵巧操作至关重要的局部力特征和净力特征。在真实世界实验中，我们的方法在四项细粒度、高接触的灵巧操作任务上取得了78%的平均成功率，与其他方法相比展现出了有效性和鲁棒性。进一步分析表明，我们的方法充分利用了3D触觉数据中的空间信息和力信息来完成任务。代码和视频可在https://3dtacdex.github.io查看。
英文摘要：
Tactile sensing plays a vital role in enabling robots to perform fine-grained, contact-rich tasks. However, the high dimensionality of tactile data, due to the large coverage on dexterous hands, poses significant challenges for effective tactile feature learning, especially for 3D tactile data, as there are no large standardized datasets and no strong pretrained backbones. To address these challenges, we propose a novel canonical representation that reduces the difficulty of 3D tactile feature learning and further introduces a force-based self-supervised pretraining task to capture both local and net force features, which are crucial for dexterous manipulation. Our method achieves an average success rate of 78% across four fine-grained, contact-rich dexterous manipulation tasks in real-world experiments, demonstrating effectiveness and robustness compared to other methods. Further analysis shows that our method fully utilizes both spatial and force information from 3D tactile data to accomplish the tasks. The codes and videos can be viewed at https://3dtacdex.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
