arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.16205cs.RO
收藏

CognitiveOS:基于大型多模态模型的系统,为任意类型机器人赋予生成式人工智能

CognitiveOS: Large Multimodal Model based System to Endow Any Type of Robot with Generative AI

Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院)

Artem Lykov, Mikhail Konenkov, Koffivi Fidèle Gbagbe, Mikhail Litvinov, Denis Davletshin, Aleksey Fedoseev, Miguel Altamirano Cabrera, Robinroy Peter, Dzmitry Tsetserukou

AI总结:

本文提出CognitiveOS,一个基于多智能体与Transformer架构的模块化操作系统,赋予各类机器人生成式AI能力,在推理任务中以77%成功率超越现有系统,并具备高适应性和可扩展性。

摘要:

本文介绍了CognitiveOS,这是首个专为认知机器人设计的操作系统,能够在多种机器人平台上运行。CognitiveOS采用多智能体系统架构,由基于Transformer架构的模块组成,通过内部独白格式进行通信。这些模块共同赋予机器人处理复杂现实世界任务的能力。本文详细阐述了该系统的运行原理,并描述了其九个不同模块。模块化设计使该系统相比传统的端到端方法具有独特优势,尤其是在适应性和可扩展性方面。系统的模块可根据任务需求进行配置、修改或停用,同时新模块可以无缝集成。该系统为认知机器人领域的研究人员和开发者提供了基础资源,减轻了从零构建认知机器人系统的负担。实验结果表明,该系统在不同任务、机器人平台和模块配置下展现出先进的任务理解和适应性,凸显了其在实际应用中的潜力。此外,在推理类别中,该系统以77%的当前最高成功率超越了CognitiveDog(高出15%)和RT2(高出31%)。我们提供了用于复现CognitiveOS的代码仓库和数据集:链接将在最终提交版本中提供。

英文摘要:

This paper introduces CognitiveOS, the first operating system designed for cognitive robots capable of functioning across diverse robotic platforms. CognitiveOS is structured as a multi-agent system comprising modules built upon a transformer architecture, facilitating communication through an internal monologue format. These modules collectively empower the robot to tackle intricate real-world tasks. The paper delineates the operational principles of the system along with descriptions of its nine distinct modules. The modular design endows the system with distinctive advantages over traditional end-to-end methodologies, notably in terms of adaptability and scalability. The system's modules are configurable, modifiable, or deactivatable depending on the task requirements, while new modules can be seamlessly integrated. This system serves as a foundational resource for researchers and developers in the cognitive robotics domain, alleviating the burden of constructing a cognitive robot system from scratch. Experimental findings demonstrate the system's advanced task comprehension and adaptability across varied tasks, robotic platforms, and module configurations, underscoring its potential for real-world applications. Moreover, in the category of Reasoning it outperformed CognitiveDog (by 15%) and RT2 (by 31%), achieving the highest to date rate of 77%. We provide a code repository and dataset for the replication of CognitiveOS: link will be provided in camera-ready submission.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑