arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.16971cs.OScs.AIcs.CL
收藏

AIOS: LLM Agent操作系统

AIOS: LLM Agent Operating System

Department of Computer Science, Rutgers University(计算机科学系,罗杰斯大学)

Kai Mei, Xi Zhu, Wujiang Xu, Wenyue Hua, Mingyu Jin, Zelong Li, Shuyuan Xu, Ruosong Ye, Yingqiang Ge, Yongfeng Zhang

AI总结:

本文提出AIOS架构,通过隔离资源与LLM服务至内核并提供调度等基础服务及SDK,解决LLM智能体资源管理问题,实现高达2.1倍执行加速。

摘要:

基于LLM的智能体面临重大部署挑战,特别是在资源管理方面。允许无限制地访问LLM或工具资源,可能导致智能体的资源分配和利用效率低下,甚至产生潜在危害。此外,当前智能体设计缺乏适当的调度和资源管理机制,阻碍了并发处理并限制了整体系统效率。为了应对这些挑战,本文提出了在管理基于LLM的智能体背景下的AIOS(基于LLM的AI Agent操作系统)架构。它引入了一种新颖的架构,通过将资源和LLM特定服务从智能体应用中隔离到AIOS内核,来为基于LLM的智能体提供服务。该AIOS内核为运行时智能体提供基础服务(例如,调度、上下文管理、内存管理、存储管理、访问控制)。为了增强可用性,AIOS还包括一个AIOS SDK,这是一套专为利用AIOS内核提供的功能而设计的综合API套件。实验结果表明,使用AIOS可以为各种智能体框架构建的智能体提供高达2.1倍更快的执行速度。源代码可在https://github.com/agiresearch/AIOS获取。

英文摘要:

LLM-based intelligent agents face significant deployment challenges, particularly related to resource management. Allowing unrestricted access to LLM or tool resources can lead to inefficient or even potentially harmful resource allocation and utilization for agents. Furthermore, the absence of proper scheduling and resource management mechanisms in current agent designs hinders concurrent processing and limits overall system efficiency. To address these challenges, this paper proposes the architecture of AIOS (LLM-based AI Agent Operating System) under the context of managing LLM-based agents. It introduces a novel architecture for serving LLM-based agents by isolating resources and LLM-specific services from agent applications into an AIOS kernel. This AIOS kernel provides fundamental services (e.g., scheduling, context management, memory management, storage management, access control) for runtime agents. To enhance usability, AIOS also includes an AIOS SDK, a comprehensive suite of APIs designed for utilizing functionalities provided by the AIOS kernel. Experimental results demonstrate that using AIOS can achieve up to 2.1x faster execution for serving agents built by various agent frameworks. The source code is available at https://github.com/agiresearch/AIOS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑