HapticWAM：将想象触觉蒸馏进世界-动作模型，无需推理时触觉感知
HapticWAM: Distilling Imagined Touch into a World-Action Model without Inference-Time Tactile Sensing
查看机构详情
- Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
HapticWAM通过师生蒸馏将光学触觉信息编码进世界-动作模型，在无推理时触觉输入下实现接触丰富操作，真实任务中达到77%平均成功率。
中文摘要 AI 辅助
接触丰富的操作需要估计力、滑移和接触几何，这些在场景图像中可能仍然模糊不清。光学触觉传感器既提供接触表面的视觉观测，也提供机械测量，但从这些信号中学习提出了两个挑战：超越外观表示接触，以及将其益处转移到部署时不需要指尖观测的策略中。我们引入了HapticWAM，一种世界-动作模型，它结合了异构触觉编码、结构化接触预测和师生蒸馏。其教师模型将凝胶图像与变形、剪切、分布力、合力旋量和导出的接触状态一起编码进一个冻结的视频骨干网络中。该模型不是仅预测触觉像素，而是联合生成动作和一个描述未来事件和力学的接触包。预期接触耦合使用先前想象的包来调节注意力，在直接触觉观测不可用时保留与接触相关的输入。触觉想象蒸馏将接触未来和动作预测都转移给学生模型，该学生模型保留生成式接触头但移除其指尖输入分支。在真实世界设置中，跨三个接触丰富的拾取和放置任务，HapticWAM学生模型实现了每任务平均77%的成功率（50个起始中的41个，合并82%），在其中一个任务上达到95%，优于评估的教师和基线配置。
英文摘要
Contact-rich manipulation requires estimating forces, slip and contact geometry that can remain ambiguous in scene images. Optical tactile sensors provide both visual observations of the contact surface and mechanical measurements, yet learning from these signals raises two challenges: representing contact beyond appearance and transferring its benefits to a policy that does not require fingertip observations at deployment. We introduce HapticWAM, a world-action model that combines heterogeneous tactile encoding, structured contact prediction and teacher-student distillation. Its teacher encodes gel images together with deformation, shear, distributed forces, resultant wrench and derived contact state into a frozen video backbone. Rather than predicting tactile pixels alone, the model jointly generates actions and a contact package describing future events and mechanics. Anticipatory Contact Coupling uses the previously imagined package to condition attention, preserving a contact-related input when direct tactile observations are unavailable. Haptic-Imagination Distillation transfers both contact futures and action predictions to a student that retains the generative contact head but removes its fingertip input branches. On a real-world setup, across three contact-rich pick-and-place tasks, HapticWAM Student achieves a 77% per-task mean success rate (41 of 50 starts, 82% pooled), reaching 95% on one of the tasks, outperforming the evaluated teacher and baseline configurations.
