arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.22146cs.RO
收藏

FastVLA:面向高效视觉-语言-动作模型的任务条件与时序自适应视觉令牌压缩方法

FastVLA: Task-Conditioned and Temporally Adaptive Visual Token Compression for Efficient Vision-Language-Action Models

Colorado School of Mines(科罗拉多矿业学院);Texas A&M University(德克萨斯农工大学);Microsoft AI(微软人工智能)

Jiuyi Xu, Chengyuan Qian, Zhengzhong Tu, Yang Sui, Yangming Shi

AI总结:

FastVLA是一种无训练的视觉令牌压缩框架,通过任务-视觉注意力打分与自适应压缩率,在保持VLA模型任务成功率的同时,大幅降低仿真与真实机器人的推理延迟及计算量。

摘要:

视觉-语言-动作(VLA)模型为通用机器人操纵提供了有前景的范式,但其闭环部署常受推理效率限制:每个时间步需将高分辨率、多视角图像编码为数百个视觉令牌,这些令牌主导了Transformer的计算与延迟。基于注意力的令牌压缩是自然解决方案,但对VLA控制存在挑战:(1)决策关键令牌由任务决定,而非通用视觉显著性;(2)各步骤令牌预算差异大,静态指标与预算会浪费计算或丢弃控制关键令牌。本文提出FastVLA,这是一种无训练、任务感知且时序自适应的视觉令牌压缩框架。每个步骤中,FastVLA通过任务-视觉注意力对令牌重要性打分,并基于重要性分布均匀性(采用带在线滑动窗口校准的逆辛普森度量)自适应设置压缩率。在OpenVLA、OpenVLA-OFT、CogACT模型,LIBERO、SIMPLER数据集及Franka Research 3物理机器人上评估显示,仿真中它将LLM骨干的FLOPs降低35-59%,动作延迟降低9-32%(提速1.09×-1.46×),同时保持任务成功率,效率-性能权衡优于现有即插即用的令牌压缩/剪枝/复用基线;真实机器人上,它在成功率相当的前提下将端到端延迟降低20%(提速1.25×)。代码可在该https URL获取。

英文摘要:

Vision-Language-Action (VLA) models present a promising paradigm for general-purpose robot manipulation, but closed-loop deployment is often limited by inference efficiency: at each timestep, high-resolution, often multi-view images are encoded into hundreds of visual tokens that dominate transformer computation and latency. Token compression via attention is a natural solution but is challenging for VLA control because (1) decision-critical tokens are determined by the task, not generic visual saliency, and (2) the token budget varies substantially across steps, so static metrics and budgets waste computation or discard control-critical tokens. We introduce FastVLA, a training-free, task-aware, and temporally adaptive visual token compression framework. At each step, FastVLA scores token importance via task-to-vision attention and adaptively sets the compression rate from the importance-distribution uniformity, measured by an inverse-Simpson metric with online sliding-window calibration. We evaluate FastVLA on OpenVLA, OpenVLA-OFT, and CogACT across LIBERO and SIMPLER, and on a physical Franka Research 3 robot. In simulation, it reduces LLM-backbone FLOPs by 35-59\% and action latency by 9-32\% (a 1.09$\times$-1.46$\times$ speedup) while maintaining task success, achieving a comparable or better efficiency-performance trade-off than existing plug-and-play token compression/pruning/reuse baselines. On the real robot, it cuts action-prediction latency by 20\% (1.25$\times$ speedup) at comparable success. Code is available at https://github.com/jiuyixu25/FastVLA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑