HyperVLA:通过超网络实现视觉-语言-动作模型的高效推理
HyperVLA: Efficient Inference in Vision-Language-Action Models via Hypernetworks
AI总结:
针对VLA模型推理成本高的问题,提出基于超网络的HyperVLA架构,推理时仅激活小策略,保持性能的同时将激活参数减少90倍、推理加速120倍。
摘要:
基于具有强大泛化能力的语言和视觉基础模型,并在大规模机器人数据上训练,视觉-语言-动作(VLA)模型最近已成为学习通用机器人策略的一种有前景的方法。然而,现有VLA的一个关键缺点是它们极高的推理成本。在本文中,我们提出HyperVLA来解决这个问题。与现有的在训练和推理期间都激活整个模型的单体VLA不同,HyperVLA采用了一种新颖的基于超网络(HN)的架构,该架构在推理期间仅激活一个小型任务特定策略,同时仍保留适应多任务行为所需的高模型容量。成功训练基于HN的VLA并非易事,因此HyperVLA包含几个关键的算法设计特征以提高其性能,包括适当利用现有视觉基础模型的先验知识、HN归一化以及动作生成策略。与单体VLA相比,HyperVLA在零样本泛化和少样本适应方面均实现了相似甚至更高的成功率,同时显著降低了推理成本。与最先进的VLA模型OpenVLA相比,HyperVLA在测试时将激活参数数量减少了90倍,并将推理速度提高了120倍。代码公开于https://github.com/MasterXiong/HyperVLA。
英文摘要:
Built upon language and vision foundation models with strong generalization ability and trained on large-scale robotic data, Vision-Language-Action (VLA) models have recently emerged as a promising approach to learning generalist robotic policies. However, a key drawback of existing VLAs is their extremely high inference costs. In this paper, we propose HyperVLA to address this problem. Unlike existing monolithic VLAs that activate the whole model during both training and inference, HyperVLA uses a novel hypernetwork (HN)-based architecture that activates only a small task-specific policy during inference, while still retaining the high model capacity needed to accommodate diverse multi-task behaviors during training. Successfully training an HN-based VLA is nontrivial so HyperVLA contains several key algorithm design features that improve its performance, including properly utilizing the prior knowledge from existing vision foundation models, HN normalization, and an action generation strategy. Compared to monolithic VLAs, HyperVLA achieves a similar or even higher success rate for both zero-shot generalization and few-shot adaptation, while significantly reducing inference costs. Compared to OpenVLA, a state-of-the-art VLA model, HyperVLA reduces the number of activated parameters at test time by $90\times$, and accelerates inference speed by $120\times$. Code is publicly available at https://github.com/MasterXiong/HyperVLA
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。