语义、工作流与基础设施：理解生产规模下的智能体服务

中文摘要 AI 辅助

大语言模型（LLM）智能体通过包含工具调用和用户交互的推理请求工作流来执行应用程序。在生产规模下服务这些应用程序，需要理解应用程序行为如何影响推理需求，并指导高效执行。近期的特征化研究提供了请求级的工作负载测量和智能体执行分析。然而，连接任务启动、工作流执行和推理基础设施的端到端视图仍未得到探索。本文分析了一个大规模通用智能体生产平台中两周内1170万条请求的追踪数据，该平台由包含超过1万块GPU的推理基础设施支撑。我们从三个关联层面刻画该平台：任务级启动语义、工作流级执行模式以及基础设施级服务需求。我们的测量揭示了诸如跨会话请求量高度偏斜、逻辑兄弟请求间罕见执行重叠以及跨任务边界的上下文复用等工作负载模式。基于这些观察，我们分析了部署影响并识别了开放问题，以指导智能体服务系统的未来研究。

英文摘要

Large language model (LLM) agents execute applications through a workflow of inference requests with tool calls and user interactions. Serving these applications at production scale requires understanding how application behavior shapes inference demand and for guiding efficient execution. Recent characterization studies provide request-level workload measurements and agent execution analysis. However, an end-to-end view connecting task initiation, workflow execution, and inference infrastructure remains unexplored. In this paper, we analyze a two-week trace of 11.7 million requests from a large-scale production platform for general-purpose agents, backed by inference infrastructure comprising over 10k GPUs. We characterize the platform at three connected levels: task-level initiation semantics, workflow-level execution patterns, and infrastructure level serving demands. Our measurements reveal workload patterns such as highly skewed request volumes across sessions, rare execution overlap among logical sibling requests, and context reuse across task boundaries. Building on these observations, we analyze deployment implications and identify open problems to guide future research on agent serving systems.

发表机构

• The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
• Ant Group（蚂蚁集团）

机构由 AI 辅助整理，请以论文原文为准。