基于机载视觉语言模型的多智能体机器人控制
Multi-Agent Robotic Control with Onboard Vision-Language Models
AI总结:
研究针对视觉语言模型用于机器人控制的挑战,提出多智能体系统架构,在机载硬件部署智能体,用紧凑型VLMs并经微调及新型编排智能体,经硬件在环模拟验证,证明该机载架构可行高效,有实际应用潜力且模拟环境已开源。
摘要:
视觉语言模型(VLMs)和视觉语言动作(VLA)模型在机器人控制中展现出潜力,但在可解释性、泛化性和计算需求方面面临重大挑战。本文提出一种多智能体系统(MAS)架构,通过在机载硬件上部署专门智能体来解决这些限制,消除对外部计算的依赖。该系统在模拟工业仓库中控制多功能自主移动操纵器,完成安全检查、仓库维护、仓库搜索、包裹质量验证和响应人类请求五类任务。全程使用紧凑型VLMs(3 - 20B参数)并进行微调以提高包裹检查精度,新型“Megamind”编排智能体缓解了较小模型在长期规划中固有的上下文保留问题。通过硬件在环模拟验证,结果表明完全机载的MAS架构是依赖云部署的可行且经济高效的替代方案,具有很强的实际应用潜力,模拟环境已开源。
英文摘要:
Vision Language Models (VLMs) and Vision Language Action (VLA) models have shown promise in robotic control. Yet, they face significant challenges regarding explainability, generalization, and compute requirements. This paper presents a Multi-Agent System (MAS) architecture that addresses these limitations by deploying specialized agents on onboard hardware - eliminating dependence on external compute. The system controls a multi-purpose autonomous mobile manipulator in a simulated industrial warehouse, fulfilling five task categories: safety inspection, warehouse maintenance, warehouse search, package quality verification, and responding to human requests. Compact VLMs (3-20B parameters) are used throughout, with fine-tuning applied to improve package inspection accuracy. A novel "Megamind" orchestration agent mitigates context retention issues inherent to long-horizon planning with smaller models. The system was validated in a hardware-in-the-loop simulation using an AMD Ryzen(TM) AI mini PC. Results demonstrate that a fully onboard MAS architecture is a viable, cost-efficient alternative to cloud-dependent deployments, with strong potential for real-world transfer. The simulation environment has been released as open source under the Apache 2.0 licence.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。