AT-VLA: 用于增强视觉-语言-动作模型反馈反应的自适应触觉注入
AT-VLA: Adaptive Tactile Injection for Enhanced Feedback Reaction in Vision-Language-Action Models
AI总结：
本文提出AT-VLA，一种自适应触觉注入机制，通过动态决定触觉注入的时间和位置，减少对预训练表示的干扰，同时引入触觉反应双流机制，实现快速准确的触觉响应，以提高视觉-语言-动作模型在接触丰富操作任务中的表现。
摘要：
视觉-语言-动作（VLA）模型在增强机器人代理执行多样化任务的能力方面取得了显著进展；然而，它们仍然面临在需要精确物理交互的接触丰富操作场景中的挑战。为了解决这一限制，最近的研究尝试在下游任务中整合触觉信号，使预训练的VLA能够解释触觉反馈。然而，在微调过程中引入新的模态，这些模态在预训练阶段很少出现，可能会破坏VLA的预训练能力。此外，VLA固有的缓慢推理速度会阻碍实时响应，并限制触觉反馈在动作调整中的有效利用。为克服这些挑战，我们提出了自适应触觉视觉-语言-动作（AT-VLA），引入了新颖的自适应触觉注入机制。该机制动态确定触觉注入的合适时间和位置，在显著促进动作生成时才进行注入，从而最小化对预训练表示的干扰。此外，为了实现快速准确的触觉响应，我们提出了触觉反应双流机制，将感知处理分为一个慢的视觉-语言流用于低频感知推理和一个快的触觉控制流用于高频物理交互理解，从而在0.04秒内实现实时闭环响应。现实世界实验彻底验证了AT-VLA在接触丰富操作任务中的有效性。项目页面可在：https://sites.google.com/view/at-vla。
英文摘要：
Vision-Language-Action (VLA) models have significantly advanced the capabilities of robotic agents in executing diverse tasks; however, they still face challenges in contact-rich manipulation scenarios that require precise physical interactions. To address this limitation, recent studies have attempted to incorporate tactile signals during downstream tasks, enabling pretrained VLAs to interpret tactile feedback. Nevertheless, introducing new modalities during finetuning, which are rarely present in the pretrain stage, may disrupt the pretrained capabilities of VLAs. In addition, the inherently slow inference speed of VLAs hampers real-time responsiveness and limits the effective utilization of tactile feedback for action adjustment. To overcome these challenges, we propose Adaptive Tactile Vision-Language-Action (AT-VLA), which introduces a novel Adaptive Tactile Injection mechanism. This mechanism dynamically determines the appropriate timing and locations for tactile injection, incorporating only when it significantly contributes to action generation, thereby minimizing interference with pretrained representations. Furthermore, to enable rapid and accurate tactile responses, we propose a Tactile Reaction Dual-Stream mechanism, which decouples sensory processing into a slow visual-language stream for low-frequency perceptual reasoning and a fast tactile control stream for high-frequency physical interaction understanding, achieving real-time close-loop responses within 0.04 s. Real-world experiments thoroughly validate the effectiveness of AT-VLA in contact-rich manipulation tasks. The project page is available at: https://sites.google.com/view/at-vla.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
