触觉模态融合用于视觉-语言-动作模型
Tactile Modality Fusion for Vision-Language-Action Models
AI总结：
本文提出TacFiLM，一种轻量级模态融合方法，将视觉-触觉信号整合到视觉-语言-动作（VLA）模型中。通过特征级线性调制（FiLM）对中间视觉特征进行条件化，提升接触丰富操作行为的性能。
摘要：
我们提出TacFiLM，一种轻量级模态融合方法，将视觉-触觉信号整合到视觉-语言-动作（VLA）模型中。尽管最近的VLA模型进展引入了通用且语义基础的机器人策略，但这些模型主要依赖视觉感知。然而，单独的视觉信息无法捕捉接触丰富操作中的复杂交互动态，包括接触力、表面摩擦、刚度和剪切。尽管最近尝试将触觉信号整合到VLA模型中通常通过标记拼接或大规模预训练增加复杂性，但行为模型的高计算需求需要更轻量的融合策略。为了解决这些挑战，TacFiLM提出一种训练后微调方法，通过特征级线性调制（FiLM）将预训练的触觉表示条件化到中间视觉特征上。在插入任务上的实验结果表明，在分布内和分布外任务中，成功率、直接插入性能、完成时间和力稳定性均有所提升。这些结果支持我们的方法作为整合触觉信号到VLA模型的有效方法，从而改善接触丰富操作行为。
英文摘要：
We propose TacFiLM, a lightweight modality-fusion approach that integrates visual-tactile signals into vision-language-action (VLA) models. While advances in VLAs have introduced robot policies that are both generalizable and semantically grounded, these models mainly rely on vision-based perception. Vision alone, however, cannot capture the complex interaction dynamics that occur during contact-rich manipulation, including contact forces, surface friction, compliance, and shear. While recent attempts to integrate tactile signals into VLA models often increase complexity through token concatenation or large-scale pretraining, the heavy computational demands of behaviour models necessitate lightweight fusion strategies. To address these challenges, TacFiLM outlines a post-training finetuning approach that conditions intermediate visual features on pretrained tactile representations using feature-wise linear modulation (FiLM). Experimental results on insertion and drawer opening tasks demonstrate consistent improvements in success rate, direct task performance, completion time, and force stability across both in-distribution and out-of-distribution tasks. Together, these results support our method as an effective approach to integrating tactile signals into VLA models, improving contact-rich manipulation behaviours. Project page: https://charliem7.github.io/projects/TacFilm/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
