发表机构
School of Artificial Intelligence, University of Chinese Academy of Sciences; Institute of Automation, Chinese Academy of Sciences; Zhongguancun Academy; Dexmal（中国科学院大学人工智能学院; 中国科学院自动化研究所; 中关村科学城; 德克斯马尔）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
为解决触觉感知融入视觉语言动作模型时的模态坍塌问题，提出ResTacVLA，将触觉数据转为残差触觉表示，过滤视觉可预测动态，经矢量量化瓶颈离散化，利用视觉先验不确定性自适应控制触觉整合。
AI 中文摘要
触觉感知对丰富接触操作不可或缺，但将其集成到视觉语言动作（VLA）模型中通常会导致模态坍塌，即高带宽视觉特征掩盖稀疏触觉线索。受预测编码启发，我们提出ResTacVLA。我们将触觉数据重新表述为捕获视觉先验与物理感觉之间差异的残差触觉表示。通过过滤掉视觉上可预测的动态，这种表述将稀疏触觉信号转换为密集的、高价值的信息增益，从而固有地解决带宽不匹配问题。这些残差通过矢量量化（VQ）瓶颈离散化为潜在接触原语，捕获视觉错过的关键事件。类似于神经惊奇信号，我们利用视觉先验的不确定性来自适应地控制触觉整合，在视觉不可靠阶段特别优先考虑残差，以明确防止视觉主导。实验结果表明，ResTacVLA在各种丰富接触操作任务上始终优于所有基线，同时对意外动态干扰保持稳健。项目页面：此https URL
英文摘要
Tactile perception is indispensable for contact-rich manipulation, yet integrating it into Vision-Language-Action (VLA) models often induces modality collapse, where high-bandwidth visual features overshadow sparse tactile cues. Inspired by Predictive Coding, a neural mechanism where the brain attenuates predictable inputs to prioritize surprising stimuli, we propose ResTacVLA. Rather than treating tactile data as raw input, we reformulate it as a Residual Tactile Representation capturing the discrepancy between visual priors and physical sensations. By filtering out visually predictable dynamics, this formulation transforms sparse tactile signals into dense, high-value information gain, thereby inherently resolving the bandwidth mismatch. These residuals are discretized through a Vector Quantized (VQ) bottleneck into Latent Contact Primitives that capture critical events missed by vision. Analogous to the neural surprise signal, we leverage the uncertainty of the visual prior to adaptively gate tactile integration, prioritizing residuals specifically during visually unreliable phases to explicitly prevent visual dominance. Experimental results show that ResTacVLA consistently outperforms all baselines on a diverse set of contact-rich manipulation tasks, while remaining robust to unexpected dynamic disturbances. Project page: https://awilekong.github.io/ResTacVLA/
Comments8 pages, 6 figures, 3 tables. Accepted by IROS 2026, Project page: https://awilekong.github.io/ResTacVLA/
