学习感知未来：DreamTacVLA用于接触丰富的 manipulation
Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation
AI总结：
DreamTacVLA通过结合高分辨率触觉图像与多尺度视觉信息，提升接触丰富任务的鲁棒性，实现95%的成功率。
摘要：
视觉-语言-动作（VLA）模型通过映射网络级知识到机器人控制展现出显著泛化能力，但仍然无法感知物理接触。因此，它们在需要推理力、纹理和滑动的接触丰富 manipulation 任务中表现不佳。尽管一些方法整合了低维触觉信号，但未能捕捉到此类交互所需的高分辨率动态。为了解决这一限制，我们引入了 DreamTacVLA，通过学习感知未来来将 VLA 模型扎根于接触物理。我们的模型采用分层感知方案，其中高分辨率触觉图像作为微视觉输入，结合腕部相机局部视觉和第三人称宏观视觉。为了协调这些多尺度感官流，我们首先训练一个统一的策略，使用分层空间对齐（HSA）损失将触觉标记与腕部和第三人称视图中的空间对应物对齐。为了进一步加深模型对细粒度接触动态的理解，我们使用触觉世界模型进行微调，预测未来的触觉信号。为了缓解触觉数据稀缺和触觉传感器易磨损的性质，我们构建了一个混合大规模数据集，来源包括高保真数字孪生和现实世界实验。通过预测即将到来的触觉状态，DreamTacVLA获得丰富的接触物理模型，并根据真实观察和想象的后果来条件化其动作。在接触丰富的 manipulation 任务中，它优于最先进的 VLA 基线，达到95%的成功率，突显了理解物理接触对于鲁棒、触觉感知的机器人代理的重要性。
英文摘要：
Vision-Language-Action (VLA) models have shown remarkable generalization by mapping web-scale knowledge to robotic control, yet they remain blind to physical contact. Consequently, they struggle with contact-rich manipulation tasks that require reasoning about force, texture, and slip. While some approaches incorporate low-dimensional tactile signals, they fail to capture the high-resolution dynamics essential for such interactions. To address this limitation, we introduce DreamTacVLA, a framework that grounds VLA models in contact physics by learning to feel the future. Our model adopts a hierarchical perception scheme in which high-resolution tactile images serve as micro-vision inputs coupled with wrist-camera local vision and third-person macro vision. To reconcile these multi-scale sensory streams, we first train a unified policy with a Hierarchical Spatial Alignment (HSA) loss that aligns tactile tokens with their spatial counterparts in the wrist and third-person views. To further deepen the model's understanding of fine-grained contact dynamics, we finetune the system with a tactile world model that predicts future tactile signals. To mitigate tactile data scarcity and the wear-prone nature of tactile sensors, we construct a hybrid large-scale dataset sourced from both high-fidelity digital twin and real-world experiments. By anticipating upcoming tactile states, DreamTacVLA acquires a rich model of contact physics and conditions its actions on both real observations and imagined consequences. Across contact-rich manipulation tasks, it outperforms state-of-the-art VLA baselines, achieving up to 95% success, highlighting the importance of understanding physical contact for robust, touch-aware robotic agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
