发表机构
Tsinghua University; Beihang University; Communication University of China; The University of Hong Kong（清华大学; 北京航空航天大学; 中国传媒大学; 香港大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对人形机器人移动操作中物理接触感知不足的问题，提出Uni-VLaT方法，通过引入预测性触觉通路整合全身体触觉信息到VLA策略，显著提升接触丰富任务的成功率。
AI 中文摘要
物理接触通常决定人形机器人在移动操作过程中应如何响应，然而仅依靠视觉和本体感觉往往不足以表征物理交互，尤其是在接触区域被遮挡的情况下。与在预定义区域进行的稀疏力或力矩测量不同，分布式触觉传感能够保留机器人身体上空间分辨的接触模式。因此，我们研究如何将这种全身体触觉信息整合到视觉-语言-动作（VLA）策略中，以用于接触丰富的控制任务。我们的方法Uni-VLaT引入了一个触觉通路，其潜在状态不仅用于动作生成，还用于预测未来的触觉、本体感觉和视觉表征。这种预测性目标构建了以触觉为中心的多模态上下文，促进了对物理世界更结构化的理解。我们在五个真实机器人任务上评估了Uni-VLaT，涵盖触觉触发的移动、持续物理交互、人机接触和移动操作。Uni-VLaT实现了75%的平均成功率，比没有触觉输入的基线高出43个百分点，比有触觉输入但无预测监督的基线高出7个百分点。在两个预训练的VLA骨干网络上，我们的方法在桌面清扫任务上均提升了30个百分点，在背部轻拍行走任务上提升了85-90个百分点。消融研究进一步表明，情境化的触觉预测和绝对未来目标对性能至关重要。这些结果表明，预测性触觉学习为将预训练的VLA策略扩展到全身体物理交互提供了一条有效途径。
英文摘要
Physical contact often determines how a humanoid should respond during loco-manipulation, yet vision and proprioception alone are often insufficient to characterize physical interaction, especially when the contact region is occluded. Unlike sparse force or torque measurements at predefined regions, distributed tactile sensing preserves spatially resolved contact patterns across the robot body. We therefore study how to integrate such whole-body tactile information into vision-language-action (VLA) policies for contact-rich control. Our approach, Uni-VLaT, introduces a tactile pathway whose latent state is trained not only for action generation, but also to predict future tactile, proprioceptive, and visual representations. This predictive objective builds a tactile-anchored multimodal context, encouraging a more structured understanding of the physical world. We evaluate Uni-VLaT on five real-robot tasks covering tactile-triggered locomotion, sustained physical interaction, human-robot contact, and loco-manipulation. Uni-VLaT achieves a 75% average success rate, outperforming a baseline without tactile input by 43 points and a tactile-input baseline without predictive supervision by 7 points. Across two pretrained VLA backbones, our method improves Table Sweeping by 30 points on both backbones and Back-Tap Walking by 85-90 points. Ablations further show that contextualized tactile prediction and absolute future targets are critical to performance. These results indicate that predictive tactile learning provides an effective route for extending pretrained VLA policies to whole-body physical interaction.
Comments12 pages, 5 figures
