SwarmX: 面向低延迟智能体系统的智能调度
SwarmX: Agentic Scheduling for Low-Latency Agentic Systems
AI总结：
提出SwarmX系统，通过调度专用神经预测器捕获提示、设备、运行时和目标模型特征，结合路由器与缩放器实现尾延迟感知决策，在近千GPU和百万CPU核的生产部署中，相比现有调度器将尾延迟降低61.5%，吞吐量提升2倍。
摘要：
智能体AI应用组合了多次模型调用和工具执行，给GPU-CPU集群带来了新的调度挑战。其推理时间和模型调用结构通常依赖于提示语义，使得传统调度方法无法有效支持低延迟服务。本文提出SwarmX系统，实现了面向低延迟智能体应用的智能调度。SwarmX使用调度专用的神经预测器来捕获提示、设备、运行时和目标模型特征；向路由器和缩放器暴露分布预测以实现尾感知决策；并提供预测器训练和在线自适应机制。这些预测器和机制被集成到一个调度器-智能体框架中，为与现有调度和模型服务基础设施的集成提供了通用基础。我们通过生产部署（近千个GPU和一百万个CPU核心）以及在128-GPU测试台上的控制实验评估了SwarmX。在多智能体代码生成、深度研究和多模态智能体工作流中，与最先进的调度器相比，SwarmX将尾延迟降低了高达61.5%，并在相同SLO下维持了高达生产调度器2倍的吞吐量。
英文摘要：
Agentic AI applications compose multiple model calls and tool executions, creating new scheduling challenges for GPU-CPU clusters. Their inference time and model-call structure often depend on prompt semantics, making conventional scheduling approaches ineffective for low-latency serving. This paper presents SwarmX, a system that implements agentic scheduling for low-latency agentic applications. SwarmX uses scheduling-specific neural predictors to capture prompt, device, runtime, and target-model features; exposes distributional predictions to routers and scalers for tail-aware decisions; and provides mechanisms for predictor training and online adaptation. These predictors and mechanisms are integrated into a scheduler-agent framework that provides a common substrate for integration with existing scheduling and model-serving infrastructure. We evaluate SwarmX using production deployment (nearly one thousand GPUs and one million CPU cores) and controlled experiments on a 128-GPU testbed. Across multi-agent code generation, deep research, and multimodal agentic workflows, SwarmX reduces tail latency by up to 61.5% compared to state-of-the-art schedulers and sustains up to 2x the throughput of production schedulers under the same SLO.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
