Tactile-VLA：解锁 Vision-Language-Action 模型的物理知识以实现触觉泛化
Tactile-VLA: Unlocking Vision-Language-Action Model's Physical Knowledge for Tactile Generalization
AI总结：
本文提出 Tactile-VLA 框架，通过融合触觉感知、混合位置-力控制器与推理模块，将 VLA 模型的隐式知识转化为精确物理交互，实现了富接触任务中的零样本泛化。
摘要：
Vision-Language-Action (VLA) 模型凭借其视觉-语言组件丰富的隐式知识，取得了显著成就。然而，要实现通用型机器人智能体，必须将其精确地扎根于物理交互中，尤其是在需要精细力控的富接触场景中。我们将 VLA 的隐式知识从识别“做什么”推进到指导“如何与真实世界进行物理交互”。本文提出了 Tactile-VLA，一个深度融合视觉、语言、动作和触觉感知的新颖框架。该框架包含一个混合位置-力控制器，用于将模型意图转化为精确的物理动作，以及一个推理模块，使机器人能够根据触觉反馈调整其策略。实验证明了 Tactile-VLA 在三个关键方面的有效性和泛化能力：(1) 实现触觉感知指令遵循，(2) 利用与触觉相关的常识，(3) 促进涉及触觉的自适应推理。一个关键发现是，VLM 的先验知识已经包含对物理交互的语义理解；通过仅用少量演示将其与机器人的触觉传感器连接，我们就能激活这一先验知识，在富接触任务中实现零样本泛化。
英文摘要：
Vision-Language-Action (VLA) models have shown remarkable achievements, driven by the rich implicit knowledge of their vision-language components. However, achieving generalist robotic agents demands precise grounding into physical interactions, especially in contact-rich scenarios where fine-grained force control is essential. We advance VLAs' implicit knowledge beyond identifying what to do, towards guiding how to physically interact with real world. This paper introduces Tactile-VLA, a novel framework that deeply fuses vision, language, action, and tactile sensing. This framework incorporates a hybrid position-force controller to translate the model's intentions into precise physical actions and a reasoning module that allows the robot to adapt its strategy based on tactile feedback. Experiments demonstrate Tactile-VLA's effectiveness and generalizability in three key aspects: (1) enabling tactile-aware instruction following, (2) utilizing tactile-relevant commonsense, and (3) facilitating adaptive tactile-involved reasoning. A key finding is that the VLM's prior knowledge already contains semantic understanding of physical interaction; by connecting it to the robot's tactile sensors with only a few demonstrations, we can activate this prior knowledge to achieve zero-shot generalization in contact-rich tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
