SP-VLA: 一种面向VLA模型加速的模型调度与令牌剪枝联合方法
SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration
AI总结：
针对VLA模型计算成本高、难以适配实时任务的问题，提出SP-VLA框架，通过动作感知模型调度和空间-语义双感知令牌剪枝联合加速，在多个基准上实现显著无损加速且性能有提升。
摘要：
视觉-语言-动作（Vision-Language-Action，VLA）模型凭借其强大的控制能力受到越来越多的关注。然而，其高昂的计算成本和较低的执行频率使其难以适配机器人操作、自主导航等实时任务。现有的VLA加速方法主要聚焦于结构优化，忽略了这类模型运行在序贯决策环境中的事实，因此序贯动作生成中的时间冗余与视觉输入中的空间冗余问题均未得到解决。\n为此，我们提出SP-VLA，这是一个通过联合模型调度与令牌剪枝来加速VLA模型的统一框架。具体而言，我们设计了一种动作感知的模型调度机制，通过在VLA模型与轻量生成器之间动态切换来降低时间冗余。受人类“关注关键决策点、其余动作依赖直觉”的运动模式启发，我们将VLA动作分为审慎型和直觉型两类，将前者分配给VLA模型处理，后者分配给轻量生成器处理，通过模型协同调度实现频率自适应执行。\n为解决空间冗余问题，我们进一步提出了一种空间-语义双感知令牌剪枝方法。该方法将令牌分为空间型和语义型两类，并根据其双感知重要性进行剪枝，以加速VLA推理。这两种机制协同作用，引导VLA模型聚焦于关键动作和显著视觉信息，在保持高精度的同时实现有效加速。\n大量实验表明，我们的方法在LIBERO数据集上实现了1.5倍的无损加速，在SimplerEnv数据集上实现了2.4倍的无损加速，平均性能提升最高达6%。在SimplerEnv上推理频率和延迟提升了2.2倍，在LIBERO上提升了1.4倍。
英文摘要：
Vision-Language-Action (VLA) models have attracted increasing attention for their strong control capabilities. However, their high computational cost and low execution frequency hinder their suitability for real-time tasks such as robotic manipulation and autonomous navigation. Existing VLA acceleration methods primarily focus on structural optimization, overlooking the fact that these models operate in sequential decision-making environments. As a result, temporal redundancy in sequential action generation and spatial redundancy in visual input remain unaddressed. To this end, we propose SP-VLA, a unified framework that accelerates VLA models by jointly scheduling models and pruning tokens. Specifically, we design an action-aware model scheduling mechanism that reduces temporal redundancy by dynamically switching between VLA model and a lightweight generator. Inspired by the human motion pattern of focusing on key decision points while relying on intuition for other actions, we categorize VLA actions into deliberative and intuitive, assigning the former to the VLA model and the latter to the lightweight generator, enabling frequency-adaptive execution through collaborative model scheduling. To address spatial redundancy, we further develop a spatio-semantic dual-aware token pruning method. Tokens are classified into spatial and semantic types and pruned based on their dual-aware importance to accelerate VLA inference. These two mechanisms work jointly to guide the VLA in focusing on critical actions and salient visual information, achieving effective acceleration while maintaining high accuracy. Extensive experiments show that our method achieves 1.5$\times$ lossless acceleration in LIBERO and 2.4$\times$ in SimplerEnv, with up to 6% average performance gain. Inference frequency and latency improve by 2.2$\times$ in SimplerEnv and 1.4$\times$ in LIBERO.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
