arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2505.04769cs.CV

视觉-语言-动作(VLA)模型:概念、进展、应用与挑战

Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges

Cornell University(康奈尔大学);The Hong Kong University of Science and Technology(香港科学与技术大学);University of the Peloponnese(希腊皮洛斯大学)

Ranjan Sapkota, Yang Cao, Konstantinos I. Roumeliotis, Manoj Karkee

AI总结:

本文综述了视觉-语言-动作(VLA)模型的概念、进展、应用与挑战,探讨了其在自动驾驶、医疗机器人等领域的应用及未来发展方向。

摘要:

视觉-语言-动作(VLA)模型标志着人工智能领域的一次变革性进步,旨在在一个计算框架内统一感知、自然语言理解以及具身动作。本文综述全面总结了最近在VLA模型方面的进展,系统地组织在五个主题支柱中,以结构该快速发展的领域景观。我们首先建立VLA系统的概念基础,追溯其从跨模态学习架构到整合视觉语言模型(VLMs)、动作规划器和分层控制器的通用代理的发展历程。我们的方法采用严谨的文献综述框架,涵盖了过去三年内超过80个VLA模型的发表。关键进展领域包括架构创新、高效的训练策略以及实时推理加速。我们探索了多样化的应用领域,如自动驾驶、医疗和工业机器人、精准农业、人形机器人以及增强现实。我们分析了挑战并提出了解决方案,包括代理适应和跨具身规划。此外,我们概述了一个前瞻性的发展路线图,其中VLA模型、VLMs和代理AI会融合,以加强社会协调、适应性和通用的具身代理。本文工作预计将成为推进智能现实世界机器人和人工智能通用智能的基础参考。项目存储库可在GitHub上获得:https://github.com/Applied-AI-Research-Lab/Vision-Language-Action-Models-Concepts-Progress-Applications-and-Challenges。[索引术语:视觉语言动作,VLA,视觉语言模型,VLMs,动作标记化,NLP]

英文摘要:

Vision-Language-Action (VLA) models mark a transformative advancement in artificial intelligence, aiming to unify perception, natural language understanding, and embodied action within a single computational framework. This foundational review presents a comprehensive synthesis of recent advancements in Vision-Language-Action models, systematically organized across five thematic pillars that structure the landscape of this rapidly evolving field. We begin by establishing the conceptual foundations of VLA systems, tracing their evolution from cross-modal learning architectures to generalist agents that tightly integrate vision-language models (VLMs), action planners, and hierarchical controllers. Our methodology adopts a rigorous literature review framework, covering over 80 VLA models published in the past three years. Key progress areas include architectural innovations, efficient training strategies, and real-time inference accelerations. We explore diverse application domains such as autonomous vehicles, medical and industrial robotics, precision agriculture, humanoid robotics, and augmented reality. We analyzed challenges and propose solutions including agentic adaptation and cross-embodiment planning. Furthermore, we outline a forward-looking roadmap where VLA models, VLMs, and agentic AI converge to strengthen socially aligned, adaptive, and general-purpose embodied agents. This work, is expected to serve as a foundational reference for advancing intelligent, real-world robotics and artificial general intelligence. The project repository is available on GitHub as https://github.com/Applied-AI-Research-Lab/Vision-Language-Action-Models-Concepts-Progress-Applications-and-Challenges. [Index Terms: Vision Language Action, VLA, Vision Language Models, VLMs, Action Tokenization, NLP]

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑