arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.07403cs.MAcs.RO
收藏

基于机载视觉语言模型的多智能体机器人控制

Multi-Agent Robotic Control with Onboard Vision-Language Models

Faculty of Mathematics and Information Science, Warsaw University of Technology(华沙技术大学数学与信息科学学院)

Kajetan Rachwał, Maciej Majek, Bartłomiej Boczek, Jakub Matejczyk, Dominik Matejkowski, Adam Dąbrowski, Tim Seyde, Alexander Amini, Maria Ganzha

AI总结:

研究针对视觉语言模型用于机器人控制的挑战,提出多智能体系统架构,在机载硬件部署智能体,用紧凑型VLMs并经微调及新型编排智能体,经硬件在环模拟验证,证明该机载架构可行高效,有实际应用潜力且模拟环境已开源。

摘要:

视觉语言模型(VLMs)和视觉语言动作(VLA)模型在机器人控制中展现出潜力,但在可解释性、泛化性和计算需求方面面临重大挑战。本文提出一种多智能体系统(MAS)架构,通过在机载硬件上部署专门智能体来解决这些限制,消除对外部计算的依赖。该系统在模拟工业仓库中控制多功能自主移动操纵器,完成安全检查、仓库维护、仓库搜索、包裹质量验证和响应人类请求五类任务。全程使用紧凑型VLMs(3 - 20B参数)并进行微调以提高包裹检查精度,新型“Megamind”编排智能体缓解了较小模型在长期规划中固有的上下文保留问题。通过硬件在环模拟验证,结果表明完全机载的MAS架构是依赖云部署的可行且经济高效的替代方案,具有很强的实际应用潜力,模拟环境已开源。

英文摘要:

Vision Language Models (VLMs) and Vision Language Action (VLA) models have shown promise in robotic control. Yet, they face significant challenges regarding explainability, generalization, and compute requirements. This paper presents a Multi-Agent System (MAS) architecture that addresses these limitations by deploying specialized agents on onboard hardware - eliminating dependence on external compute. The system controls a multi-purpose autonomous mobile manipulator in a simulated industrial warehouse, fulfilling five task categories: safety inspection, warehouse maintenance, warehouse search, package quality verification, and responding to human requests. Compact VLMs (3-20B parameters) are used throughout, with fine-tuning applied to improve package inspection accuracy. A novel "Megamind" orchestration agent mitigates context retention issues inherent to long-horizon planning with smaller models. The system was validated in a hardware-in-the-loop simulation using an AMD Ryzen(TM) AI mini PC. Results demonstrate that a fully onboard MAS architecture is a viable, cost-efficient alternative to cloud-dependent deployments, with strong potential for real-world transfer. The simulation environment has been released as open source under the Apache 2.0 licence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑