TaF-VLA：面向力感知操控的视觉-语言-动作模型中的触觉-力对齐
TaF-VLA: Tactile-Force Alignment in Vision-Language-Action Models for Force-aware Manipulation
AI总结：
TaF-VLA通过触觉-力对齐提升视觉-语言-动作模型在力感知操控中的性能。
摘要：
视觉-语言-动作（VLA）模型近年来已涌现出强大的通用性，用于机器人操作。然而，由于其主要依赖视觉模态，它们从根本上缺乏接触丰富任务所需的物理直觉，这些任务需要精确的力调节和物理推理。现有的尝试将基于视觉的触觉传感纳入VLA模型通常将触觉输入视为辅助的视觉纹理，从而忽略了表面变形与交互动力学之间的潜在相关性。为弥合这一差距，我们提出了一种从触觉-视觉对齐到触觉-力对齐的范式转变。在此，我们介绍了TaF-VLA框架，该框架将高维触觉观测显式地根植于物理交互力中。为此，我们开发了一种自动化触觉-力数据采集设备，并整理了TaF数据集，包含超过1000万同步的触觉观测、6轴力/扭矩和矩阵力图。为了将顺序触觉观测与交互力对齐，我们方法的核心组件是触觉-力适配器（TaF-适配器），一个触觉传感器编码器，用于提取离散化的潜在信息以编码触觉观测。这种机制确保学习的表示捕捉历史依赖、抗噪声的物理动态，而不是静态的视觉纹理。最后，我们将这个力对齐的编码器整合到VLA主干中。广泛的现实世界实验表明，TaF-VLA策略在接触丰富的任务上显著优于最先进的触觉-视觉对齐和视觉-only基线，验证了其通过跨模态物理推理实现稳健、力感知操作的能力。
英文摘要：
Vision-Language-Action (VLA) models have recently emerged as powerful generalists for robotic manipulation. However, due to their predominant reliance on visual modalities, they fundamentally lack the physical intuition required for contact-rich tasks that require precise force regulation and physical reasoning. Existing attempts to incorporate vision-based tactile sensing into VLA models typically treat tactile inputs as auxiliary visual textures, thereby overlooking the underlying correlation between surface deformation and interaction dynamics. To bridge this gap, we propose a paradigm shift from tactile-vision alignment to tactile-force alignment. Here, we introduce TaF-VLA, a framework that explicitly grounds high-dimensional tactile observations in physical interaction forces. To facilitate this, we develop an automated tactile-force data acquisition device and curate the TaF-Dataset, comprising over 10 million synchronized tactile observations, 6-axis force/torque, and matrix force map. To align sequential tactile observations with interaction forces, the central component of our approach is the Tactile-Force Adapter (TaF-Adapter), a tactile sensor encoder that extracts discretized latent information for encoding tactile observations. This mechanism ensures that the learned representations capture history-dependent, noise-insensitive physical dynamics rather than static visual textures. Finally, we integrate this force-aligned encoder into a VLA backbone. Extensive real-world experiments demonstrate that TaF-VLA policy significantly outperforms state-of-the-art tactile-vision-aligned and vision-only baselines on contact-rich tasks, verifying its ability to achieve robust, force-aware manipulation through cross-modal physical reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
