UP-VLA：面向具身智能体的统一理解与预测模型
UP-VLA: A Unified Understanding and Prediction Model for Embodied Agent
AI总结：
本文提出UP-VLA模型，通过联合多模态理解与未来预测目标训练VLA，增强高级语义与低级空间理解，在Calvin ABC-D基准上提升33%，并改善真实世界操作任务成功率。
摘要：
视觉-语言-动作（VLA）模型的最新进展利用预训练的视觉-语言模型（VLM）来提升泛化能力。VLM通常在视觉-语言理解任务上进行预训练，提供丰富的语义知识和推理能力。然而，先前的研究表明，VLM通常关注高级语义内容而忽略低级特征，限制了其捕捉详细空间信息和理解物理动态的能力。这些对具身控制任务至关重要的方面，在现有预训练范式中仍未被充分探索。在本文中，我们研究了VLA的训练范式，并引入了UP-VLA，这是一个使用多模态理解和未来预测目标进行训练的统一VLA模型，同时增强了高级语义理解和低级空间理解。实验结果表明，与之前的最先进方法相比，UP-VLA在Calvin ABC-D基准上实现了33%的提升。此外，UP-VLA在现实世界操作任务中表现出更高的成功率，特别是那些需要精确空间信息的任务。
英文摘要：
Recent advancements in Vision-Language-Action (VLA) models have leveraged pre-trained Vision-Language Models (VLMs) to improve the generalization capabilities. VLMs, typically pre-trained on vision-language understanding tasks, provide rich semantic knowledge and reasoning abilities. However, prior research has shown that VLMs often focus on high-level semantic content and neglect low-level features, limiting their ability to capture detailed spatial information and understand physical dynamics. These aspects, which are crucial for embodied control tasks, remain underexplored in existing pre-training paradigms. In this paper, we investigate the training paradigm for VLAs, and introduce \textbf{UP-VLA}, a \textbf{U}nified VLA model training with both multi-modal \textbf{U}nderstanding and future \textbf{P}rediction objectives, enhancing both high-level semantic comprehension and low-level spatial understanding. Experimental results show that UP-VLA achieves a 33% improvement on the Calvin ABC-D benchmark compared to the previous state-of-the-art method. Additionally, UP-VLA demonstrates improved success rates in real-world manipulation tasks, particularly those requiring precise spatial information.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
