TacVLA: 基于接触感知的触觉融合用于鲁棒的视觉-语言-动作操控
TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulation
AI总结：
TacVLA通过引入触觉模态提升视觉-语言-动作操控的鲁棒性，采用接触感知门控机制实现多模态融合，实验显示在拆解、箱内拾取和视觉遮挡场景中性能提升显著。
摘要：
视觉-语言-动作（VLA）模型在机器人操控中表现出显著优势，但其依赖视觉和语言在涉及视觉遮挡、精细操控和物理接触的任务中表现欠佳。为解决这些问题，我们提出了TacVLA，通过将触觉模态融入基于变压器的策略中进行微调，以增强精细操控能力。具体而言，我们引入了接触感知门控机制，仅在检测到接触时激活触觉标记，实现自适应多模态融合并避免无关触觉干扰。融合的视觉、语言和触觉标记在变压器架构中联合处理，以加强接触丰富交互期间的跨模态语义对齐。在约束锁定拆解、箱内拾取和鲁棒性评估等大量实验中，我们的模型优于基线，平均在拆解任务中提升20%的成功率，在箱内拾取任务中提升60%，在视觉遮挡场景中提升2.1倍。视频可在https://sites.google.com/view/tacvla获取，代码将被发布。
英文摘要：
Vision-Language-Action (VLA) models have demonstrated significant advantages in robotic manipulation. However, their reliance on vision and language often leads to suboptimal performance in tasks involving visual occlusion, fine-grained manipulation, and physical contact. To address these challenges, we propose TacVLA, a fine-tuned VLA model by incorporating tactile modalities into the transformer-based policy to enhance fine-grained manipulation capabilities. Specifically, we introduce a contact-aware gating mechanism that selectively activates tactile tokens only when contact is detected, enabling adaptive multimodal fusion while avoiding irrelevant tactile interference. The fused visual, language, and tactile tokens are jointly processed within the transformer architecture to strengthen cross-modal grounding during contact-rich interaction. Extensive experiments on constraint-locked disassembly, in-box picking and robustness evaluations demonstrate that TacVLA outperforms baselines, %including existing VLA models and diffusion policies, improving the performance by averaging 20\% success rate in disassembly and 60\% in in-box picking, achieving a 2.1$\times$ improvement under visual occlusion, and showing recovery behavior under human disturbance. Videos are available at https://sites.google.com/view/tacvla.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
