arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.18532cs.CVcs.AIcs.RO

VLANeXt: 构建强大VLA模型的配方

VLANeXt Family: A Systematic Study of VLA Models from Core Recipes to Emerging Paradigms

S-Lab, Nanyang Technological University(南洋理工大学S实验室);SenseTime Research(商汤科技研究院);Sun Yat-sen University(中山大学);Shanghai Jiao Tong University(上海交通大学)

Xiao-Ming Wu, Kang Liao, Yihang Luo, Bin Fan, Jian-Jian Jiang, Runze Yang, Zhonghua Wu, Wei-Shi Zheng, Chen Change Loy

AI总结:

本文通过统一框架和评估设置重新审视VLA设计空间,系统分析了基础组件、感知要素和动作建模视角,总结出12项关键发现,提出了一种简单有效的VLA模型VLANeXt,并在LIBERO和LIBERO-plus基准测试中超越了现有方法,同时提供了易于使用的代码库。

摘要:

在大基础模型兴起之后,视觉-语言-动作模型(VLAs)应运而生,利用视觉语言模型的强大视觉和语言理解能力进行通用目的策略学习。然而,当前VLA领域仍处于碎片化和探索阶段。尽管许多团队提出了各自的VLA模型,但训练协议和评估设置的一致性不足,使得难以确定哪些设计选择真正重要。为了使这一发展领域更具结构化,我们重新审视VLA设计空间,基于类似RT-2的简单VLA基线,系统地分析了三个维度:基础组件、感知要素和动作建模视角。从这项研究中,我们提炼出12项关键发现,共同构成了构建强大VLA模型的实用配方。该探索的成果是一种简单而有效的模型VLANeXt,它在LIBERO和LIBERO-plus基准测试中优于现有方法,并在现实世界实验中表现出色。我们还发布了一个统一且易于使用的代码库,以重现我们的发现、探索设计空间并基于共享基础开发新的VLA变体。代码库可在https://github.com/DravenALG/VLANeXt上获得。

英文摘要:

Following the rise of large foundation models, Vision-Language-Action models (VLAs) emerged, leveraging strong visual and language understanding from Vision-Language Models for general-purpose policy learning. Yet, the current VLA landscape remains fragmented and exploratory. Although many groups have proposed their own VLA models, inconsistencies in training protocols and evaluation settings make it difficult to identify which design choices truly matter. To bring structure to this evolving space, we reexamine the VLA design space under a unified framework and evaluation setup. Starting from a simple VLA baseline similar to RT-2, which is the origin of VLA, we systematically dissect design choices along three dimensions: foundational components, perception essentials, and action modeling perspectives. From this study, we distill 12 key findings that together form a practical recipe for building strong VLA models. The outcome of this exploration is a simple yet effective model, VLANeXt. It outperforms the state-of-the-art methods on the LIBERO and LIBERO-plus benchmarks and demonstrates strong performance in real-world experiments. Beyond identifying the core recipe, we further ask how far these design principles extend to the emerging paradigms in VLAs. We thus expand VLANeXt along several emerging directions, including model scaling, latent-action pretraining, latent predictive representation learning, and world action modeling. These studies give rise to the VLANeXt family, spanning compact and scaled VLA variants, latent-action models, JEPA-style predictive models, and World Action Models. Our results show that the core recipe provides a strong foundation across different model scales and emerging paradigms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑