环境感知自适应剪枝与交错推断调度用于视觉-语言-动作模型
Environment-Aware Adaptive Pruning with Interleaved Inference Orchestration for Vision-Language-Action Models
AI总结:
EcoVLA通过环境感知自适应剪枝和交错推断调度,实现视觉-语言-动作模型的高效参数稀疏化,提升推理速度并保持高成功率。
摘要:
尽管视觉-语言-动作(VLA)模型在具身智能中具有潜力,但其庞大的参数数量导致显著的推理延迟,阻碍了实时操作,促使参数稀疏化。然而,随着VLA执行过程中环境的变化,最优稀疏模式也会相应变化。静态剪枝缺乏适应环境动态所需的适应性,而固定间隔动态层剪枝则存在粒度粗和重新训练开销高的问题。为弥合这一差距,我们提出了EcoVLA,一种无需训练的即插即用自适应剪枝框架,支持与现有VLA加速方法的正交组合。EcoVLA包含两个组成部分:环境感知自适应剪枝(EAP)和交错推断调度(I²O)。EAP是一种轻量级的自适应通道剪枝方法,结合了物理环境的时间一致性来更新稀疏模式。I²O利用VLA推理中固有的FLOPs气泡来并行调度剪枝方法,确保对延迟的影响可忽略不计。在多样化的VLA模型和基准上评估,EcoVLA实现了最先进的性能,达到1.60倍的速度提升,仅导致0.4%的成功率下降,并在结合标记剪枝时进一步达到2.18倍的速度提升,仅导致0.5%的退化。我们进一步在真实世界机器人上验证了EcoVLA的有效性。
英文摘要:
While Vision-Language-Action (VLA) models hold promise in embodied intelligence, their large parameter counts lead to substantial inference latency that hinders real-time manipulation, motivating parameter sparsification. However, as the environment evolves during VLA execution, the optimal sparsity patterns change accordingly. Static pruning lacks the adaptability required for environment dynamics, whereas fixed-interval dynamic layer pruning suffers from coarse granularity and high retraining overheads. To bridge this gap, we propose EcoVLA, a training-free, plug-and-play adaptive pruning framework that supports orthogonal combination with existing VLA acceleration methods. EcoVLA comprises two components: Environment-aware Adaptive Pruning (EAP) and Interleaved Inference Orchestration ($I^2O$). EAP is a lightweight adaptive channel pruning method that incorporates the temporal consistency of the physical environment to update sparsity patterns. $I^2O$ leverages the FLOPs bubbles inherent in VLA inference to schedule the pruning method in parallel, ensuring negligible impact on latency. Evaluated on diverse VLA models and benchmarks, EcoVLA delivers state-of-the-art performance, achieving up to 1.60$\times$ speedup with only a 0.4% drop in success rate, and further reaches 2.18$\times$ speedup with only a 0.5% degradation when combined with token pruning. We further validate the effectiveness of EcoVLA on real-world robots.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。