arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.14902cs.RO

VLA^2：赋予视觉-语言-动作模型一个用于未见概念操作的智能体框架

VLA^2: Empowering Vision-Language-Action Models with an Agentic Framework for Unseen Concept Manipulation

Zhejiang University（浙江大学）；MiLAB, Westlake University（西湖大学 MiLAB）；Southern University of Science and Technology（南方科技大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））

Han Zhao, Jiaxuan Zhang, Wenxuan Song, Pengxiang Ding, Donglin Wang

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出VLA^2智能体框架，利用外部知识增强VLA模型，解决未见对象操作泛化问题，在困难基准上成功率提升44.2%。

摘要：

当前的视觉-语言-动作（VLA）模型在大规模机器人数据上预训练后，展现出强大的多任务能力，并能很好地泛化到操作中视觉和语言指令的变化。然而，当面对训练数据之外的对象概念（如数据集中未见过的对象描述和纹理）时，它们的成功率显著下降。为解决这一问题，我们提出了一种新颖的智能体框架VLA^2，该框架利用OpenVLA作为执行主干，并有效利用外部模块（如网络检索和对象检测）为VLA提供目标对象的视觉和文本知识。这种方法缓解了处理分布外对象时的泛化失败。基于LIBERO仿真环境，我们引入了新对象和对象描述，构建了一个具有三个难度级别的新评估基准，以测试我们方法的有效性。在我们设计的困难级别泛化基准上，我们的框架成功超越了当前最先进的模型。与独立的OpenVLA基线相比，VLA^2在困难级别基准上的成功率提高了44.2%，在所有定制环境中平均提高了20.2%，且在领域内任务上没有任何性能下降。项目网站：https://vla-2.github.io。

英文摘要：

Current vision-language-action (VLA) models, pre-trained on large-scale robotic data, exhibit strong multi-task capabilities and generalize well to variations in visual and language instructions for manipulation. However, their success rate drops significantly when faced with object concepts outside the training data, such as unseen object descriptions and textures in the dataset. To address this, we propose a novel agentic framework, VLA^2, which leverages OpenVLA as the execution backbone and effectively leverages external modules such as web retrieval and object detection to provide visual and textual knowledge about target objects to the VLA. This approach mitigates generalization failure when handling out-of-distribution objects. Based on the LIBERO simulation environment, we introduced novel objects and object descriptions to construct a new evaluation benchmark with three difficulty levels to test the effectiveness of our method. Our framework successfully outperformed the current state-of-the-art models on our designed hard-level generalization benchmark. Compared to the standalone OpenVLA baseline, VLA^2 achieves a 44.2% improvement in the success rate in the hard-level benchmark and an average improvement of 20.2% in all customized environments without any performance degradation on in-domain tasks. Project website: https://vla-2.github.io.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑