面向机器人的视觉-语言-动作模型:面向实际应用综述
Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
AI总结:
本综述全面梳理视觉-语言-动作(VLA)模型在机器人领域的策略、架构、学习范式及软硬件组件,并总结平台、数据集和基准,为实际部署提供指导。
摘要:
在利用大规模语言模型(LLMs)和视觉-语言模型(VLMs)的进展推动机器人技术发展的努力日益增多的背景下,视觉-语言-动作(VLA)模型近期获得了显著关注。通过大规模统一视觉、语言和动作数据(这些数据传统上被分开研究),VLA模型旨在学习能够跨不同任务、物体、具身形态和环境进行泛化的策略。这种泛化能力预期能使机器人以最少或无需额外任务特定数据的方式解决新颖的下游任务,从而促进更灵活和可扩展的实际部署。与以往仅聚焦于动作表示或高层模型架构的综述不同,本工作提供了全面的全栈式综述,整合了VLA系统的软件和硬件组件。具体而言,本文对VLA进行了系统性综述,涵盖其策略与架构演变、架构与构建模块、模态特定处理技术以及学习范式。此外,为支持VLA在真实世界机器人应用中的部署,我们还综述了常用机器人平台、数据收集策略、公开数据集、数据增强方法和评估基准。通过这一全面综述,本文旨在为机器人社区在将VLA应用于真实世界机器人系统时提供实用指导。按训练方法、评估方法、模态和数据集分类的所有参考文献均可在我们项目网站的表格中获取:https://vla-survey.github.io 。
英文摘要:
Amid growing efforts to leverage advances in large language models (LLMs) and vision-language models (VLMs) for robotics, Vision-Language-Action (VLA) models have recently gained significant attention. By unifying vision, language, and action data at scale, which have traditionally been studied separately, VLA models aim to learn policies that generalise across diverse tasks, objects, embodiments, and environments. This generalisation capability is expected to enable robots to solve novel downstream tasks with minimal or no additional task-specific data, facilitating more flexible and scalable real-world deployment. Unlike previous surveys that focus narrowly on action representations or high-level model architectures, this work offers a comprehensive, full-stack review, integrating both software and hardware components of VLA systems. In particular, this paper provides a systematic review of VLAs, covering their strategy and architectural transition, architectures and building blocks, modality-specific processing techniques, and learning paradigms. In addition, to support the deployment of VLAs in real-world robotic applications, we also review commonly used robot platforms, data collection strategies, publicly available datasets, data augmentation methods, and evaluation benchmarks. Throughout this comprehensive survey, this paper aims to offer practical guidance for the robotics community in applying VLAs to real-world robotic systems. All references categorized by training approach, evaluation method, modality, and dataset are available in the table on our project website: https://vla-survey.github.io .
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。