TacWAM：锚点引导的力学感知触觉世界动作模型
TacWAM: Anchor-Guided World Action Model with Mechanics-Aware Tactile Prediction
浏览论文内容
中文总结 AI 辅助
该研究提出TacWAM，通过空间对齐融合触觉编码器、触觉历史编码器及锚点引导三模态注意力，在四项真实接触操作任务上使平均成功率达75.0%，远超最强基线37.5个百分点。
中文摘要 AI 辅助
世界动作模型（WAM）将未来状态预测与机器人动作生成相结合，但现有方法大多依赖视觉未来预测。视觉预测能捕捉场景结构和物体运动，却无法为接触丰富的操作过程中的力、变形、剪切和滑动提供充分监督，这提出了两项设计要求：触觉未来应承载有意义的物理信息，且不应成为动作生成的特权线索。我们提出TacWAM，一种力学感知的触觉WAM，通过三步解决该挑战：第一，空间对齐融合（SAF）触觉编码器将触觉外观、密集力场和变形流映射到共享的潜在预测空间，通过双向力和力矩重建保留全局接触信息；第二，触觉历史编码器提供时间上下文，使未来触觉预测能反映当前触觉观测之外的力和变形变化；第三，锚点引导的三模态（AGT）注意力分离当前视觉与触觉锚点、未来预测令牌及动作令牌，让未来触觉状态可监督训练，同时不被动作分支直接读取。我们在四项真实世界接触丰富的操作任务上评估TacWAM，涵盖易碎物体抓取、持续表面接触和动态手中操作。TacWAM的平均成功率达75.0%，比评估中最强基线高出37.5个百分点。阶段性消融实验显示，移除触觉历史或放松未来预测目标访问时，性能会持续下降。这些结果表明，当结合信息丰富的触觉表示和部署一致的信息约束时，未来触觉监督可提升接触感知的动作学习效果。
英文摘要
World Action Models (WAMs) combine future-state prediction with robot action generation, but existing approaches largely rely on visual futures. Visual prediction captures scene structure and object motion, yet provides limited supervision for force, deformation, shear, and slip during contact-rich manipulation. This creates two design requirements: tactile futures should carry meaningful physical information, and they should not become privileged cues for action generation. We present TacWAM, a mechanics-aware tactile WAM that addresses this challenge in three steps. First, a Spatially Aligned Fusion (SAF) Tactile Encoder maps tactile appearance, dense force fields, and deformation flow into a shared latent prediction space, with bilateral force and torque reconstruction preserving global contact information. Second, a tactile history encoder provides temporal context so future tactile prediction reflects how force and deformation change beyond the current tactile observation. Third, Anchor-Guided Tri-Modal (AGT) Attention separates current visual and tactile anchors, future prediction tokens, and action tokens, allowing future tactile states to supervise training without being directly read by the action branch. We evaluate TacWAM on four real-world contact-rich manipulation tasks covering fragile grasping, sustained surface contact, and dynamic in-hand manipulation. TacWAM achieves an average success rate of 75.0%, exceeding the strongest evaluated baseline by 37.5 percentage points. Staged ablations show consistent degradation when tactile history is removed and access to future prediction targets is relaxed. These results indicate that future tactile supervision can improve contact-aware action learning when combined with informative tactile representations and deployment-consistent information constraints.
