我如何快速跑动我的VLA?通过VLA-Perf解密VLA推理性能
How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf
AI总结:
本文通过VLA-Perf分析VLA模型的推理性能,探讨了模型设计和部署对实时推理的影响,提出了15项关键结论以指导未来VLA系统设计。
摘要:
视觉-语言-动作(VLA)模型最近在各种具身AI任务中展示了令人印象深刻的性能。然而,将VLA模型部署到现实世界机器人上会施加严格的实时推理约束,由于模型架构和推理系统的庞大组合空间,VLA的推理性能景观仍缺乏理解。在本文中,我们提出了一个根本性的研究问题:如何设计未来的VLA模型和系统以支持实时推理?为了回答这个问题,我们首先介绍了VLA-Perf,一个分析性能模型,可以分析任意组合的VLA模型和推理系统的推理性能。利用VLA-Perf,我们进行了首次系统的VLA推理性能景观研究。从模型设计的角度,我们考察了推理性能如何受到模型扩展、模型架构选择、长上下文视频输入、异步推理和双系统模型流水线的影响。从部署的角度,我们分析了VLA推理应该在哪里执行——在设备上、在边缘服务器上还是在云中,并且如何硬件能力和网络性能共同决定端到端延迟。通过从我们的全面评估中提炼出15个关键结论,我们希望这项工作能为未来VLA模型和推理系统的设计提供实用指导。
英文摘要:
Vision-Language-Action (VLA) models have recently demonstrated impressive capabilities across various embodied AI tasks. While deploying VLA models on real-world robots imposes strict real-time inference constraints, the inference performance landscape of VLA remains poorly understood due to the large combinatorial space of model architectures and inference systems. In this paper, we ask a fundamental research question: How should we design future VLA models and systems to support real-time inference? To address this question, we first introduce VLA-Perf, an analytical performance model that can analyze inference performance for arbitrary combinations of VLA models and inference systems. Using VLA-Perf, we conduct the first systematic study of the VLA inference performance landscape. From a model-design perspective, we examine how inference performance is affected by model scaling, model architectural choices, long-context video inputs, asynchronous inference, and dual-system model pipelines. From the deployment perspective, we analyze where VLA inference should be executed -- on-device, on edge servers, or in the cloud -- and how hardware capability and network performance jointly determine end-to-end latency. By distilling 15 key takeaways from our comprehensive evaluation, we hope this work can provide practical guidance for the design of future VLA models and inference systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。