PTLD: 从仿真到现实的触觉潜在知识蒸馏用于灵巧操作
PTLD: Sim-to-real Privileged Tactile Latent Distillation for Dexterous Manipulation
AI总结：
提出PTLD方法，通过真实世界触觉策略数据蒸馏鲁棒状态估计器，解决触觉仿真困难问题，在灵巧操作任务中相比纯本体感策略提升182%和57%。
摘要：
触觉灵巧操作对于自动化复杂家务任务至关重要，但学习有效控制策略仍然是一个挑战。虽然最近的工作依赖于模仿学习，但通过机器人遥操作或动觉教学获取多指手的高质量演示是困难的。另一种方法是，通过强化学习我们可以在仿真中学习技能，但快速且真实的触觉观测仿真具有挑战性。为了弥合这一差距，我们引入了PTLD：从仿真到现实的触觉潜在知识蒸馏，这是一种无需触觉仿真即可学习触觉操作技能的新方法。我们的关键思想不是模拟触觉传感器或纯粹依赖本体感策略进行零样本从仿真到现实的迁移，而是利用现实世界中的特权传感器收集真实的触觉策略数据。然后，这些数据用于蒸馏一个鲁棒的状态估计器，该估计器基于触觉输入运行。我们的实验表明，PTLD可以通过结合触觉感知显著改善在仿真中训练的本体感操作策略。在基准的掌内旋转任务中，PTLD相比纯本体感策略实现了182%的提升。我们还展示了PTLD能够学习具有挑战性的触觉掌内重定向任务，在该任务中，我们观察到达到的目标数量相比仅使用本体感提高了57%。网站：此 https URL。
英文摘要：
Tactile dexterous manipulation is essential to automating complex household tasks, yet learning effective control policies remains a challenge. While recent work has relied on imitation learning, obtaining high quality demonstrations for multi-fingered hands via robot teleoperation or kinesthetic teaching is prohibitive. Alternatively, with reinforcement we can learn skills in simulation, but fast and realistic simulation of tactile observations is challenging. To bridge this gap, we introduce PTLD: sim-to-real Privileged Tactile Latent Distillation, a novel approach to learning tactile manipulation skills without requiring tactile simulation. Instead of simulating tactile sensors or relying purely on proprioceptive policies to transfer zero-shot sim-to-real, our key idea is to leverage privileged sensors in the real world to collect real-world tactile policy data. This data is then used to distill a robust state estimator that operates on tactile input. We demonstrate from our experiments that PTLD can be used to improve proprioceptive manipulation policies trained in simulation significantly by incorporating tactile sensing. On the benchmark in-hand rotation task, PTLD achieves a 182% improvement over a proprioception only policy. We also show that PTLD enables learning the challenging task of tactile in-hand reorientation where we see a 57% improvement in the number of goals reached over using proprioception alone. Website: https://akashsharma02.github.io/ptld-website/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
