通过自适应视觉令牌缓存学习加速视觉-语言-动作模型
Learning to Accelerate Vision-Language-Action Models through Adaptive Visual Token Caching
AI总结:
本文提出通过自适应视觉令牌缓存学习加速VLA模型,提升推理效率并提高任务成功率。
摘要:
视觉-语言-动作(VLA)模型在机器人操作任务中展现了显著的泛化能力,但其显著的计算开销仍然是实际部署中的关键障碍。因此,提高推理效率对于实际的机器人应用至关重要。现有的加速方法通常依赖于启发式或静态策略——例如基于规则的令牌缓存或剪枝——这些策略与任务目标脱节,无法适应动态场景变化。在本文中,我们将推理加速重新表述为一个可学习的策略优化问题,并提出一个新颖的框架,将动态、任务感知的决策过程直接整合到VLA模型中。其核心是两个轻量级、协作的模块:一个缓存令牌选择器,用于确定哪些令牌应被重用,以及一个缓存比率预测器,用于控制应重用多少令牌。训练这些模块具有挑战性,因为它们的决策是离散的。我们通过采用可微松弛方法来解决这一问题,允许基于梯度的端到端优化。在LIBERO和SIMPLER基准以及真实机器人评估中,广泛实验表明,我们的方法在LIBERO上实现了1.76倍的时钟推理加速,同时将平均成功率提高了1.9个百分点(从75.0%提高到76.9%),并在真实任务中提高了5.0个百分点,显著优于现有基线。这项工作展示了学习任务感知计算分配策略的潜力,为既强大又高效的VLA模型铺平了道路。
英文摘要:
Vision-Language-Action (VLA) models have demonstrated remarkable generalization capabilities in robotic manipulation tasks, yet their substantial computational overhead remains a critical obstacle to real-world deployment. Improving inference efficiency is therefore essential for practical robotic applications. Existing acceleration methods often rely on heuristic or static strategies--such as rule-based token caching or pruning--that are decoupled from task objectives and fail to adapt to dynamic scene changes. In this work, we reformulate inference acceleration as a learnable policy optimization problem and propose a novel framework that integrates a dynamic, task-aware decision-making process directly into the VLA model. At its core are two lightweight, cooperative modules: a Cached Token Selector, which determines which tokens should be reused, and a Cache Ratio Predictor, which controls how many tokens to reuse. Training these modules is non-trivial due to their discrete decisions. We address this by adopting a differentiable relaxation that allows gradient-based end-to-end optimization. Extensive experiments on the LIBERO and SIMPLER benchmarks, as well as real-robot evaluations, show that our method achieves a 1.76x wall-clock inference speedup while simultaneously improving the average success rate by 1.9 percentage points (from 75.0% to 76.9%) on LIBERO and by 5.0 percentage points on real-world tasks, significantly outperforming existing baselines. This work highlights the potential of learning task-aware computational allocation policies, paving the way for VLA models that are both powerful and efficient.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。