arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.03380cs.ROcs.AI
收藏

LiteVLA-Edge: 量化在设备上多模态控制用于嵌入式机器人

LiteVLA-Edge: Quantized On-Device Multimodal Control for Embedded Robotics

Clark Atlanta University(克拉克阿特兰大学);Siemens Corporation(西门子公司)

Justin Williams, Kishor Datta Gupta, Roy George, Mrinmoy Sarkar

AI总结:

LiteVLA-Edge通过量化和GPU加速,在嵌入式机器人中实现低延迟的多模态控制,提供模块化本地执行方案。

摘要:

视觉-语言-动作(VLA)模型提供了一个统一的框架,用于感知、语言条件和动作生成,但许多现有系统仍然难以在嵌入式机器人环境中部署,因为它们的计算需求和推理延迟。在本文中,我们提出了LiteVLA-Edge,一种面向部署的VLA流水线,用于在Jetson Orin类硬件上实现完全在设备上的推理。我们的方法结合了FP32的监督图像到动作微调,以及后训练4位GGUF量化和通过llama.cpp运行时的GPU加速推理。在我们的部署配置下,LiteVLA-Edge实现了平均端到端延迟为150.5毫秒(约6.6赫兹),并在ROS 2集成的感知-推理-动作流水线中完全离线运行。而不是引入新的策略目标,我们的贡献是提供一个实用的系统路径,以在嵌入式硬件上执行紧凑的多模态控制模型,同时保持感知、推理和执行之间的模块化接口。这些结果为反应性语言条件控制提供了时间可行性,并为未来在机器人中评估在设备上的VLA提供了可重复的基准。

英文摘要:

Vision-Language-Action (VLA) models provide a unified framework for perception, language conditioning, and action generation, but many existing systems remain difficult to deploy in embedded robotic settings because of their computational requirements and inference latency. In this paper, we present LiteVLA-Edge, a deployment-oriented VLA pipeline for fully on-device inference on Jetson Orin-class hardware. Our approach combines supervised image-to-action fine-tuning in FP32 with post-training 4-bit GGUF quantization and GPU-accelerated inference through the \texttt{llama.cpp} runtime. Under our deployment configuration, LiteVLA-Edge achieves a mean end-to-end latency of 150.5\,ms (approximately 6.6\,Hz) while operating entirely offline within a ROS~2-integrated perception--reasoning--action pipeline. Rather than introducing a new policy objective, our contribution is a practical systems path for executing compact multimodal control models locally on embedded hardware while preserving modular interfaces between perception, reasoning, and actuation. These results establish timing feasibility for reactive language-conditioned control and provide a reproducible baseline for future task-level evaluation of on-device VLAs in robotics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑