发表机构
University of Chicago（芝加哥大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出SEIS，一种基于智能体自进化的推理系统优化方法，端到端优化整个mini-sglang引擎，在H100上实现3.27倍吞吐量提升，并超越现有最先进引擎，证明自进化可优化复杂系统。
AI 中文摘要
推理系统决定了语言模型的服务速度和成本，因此使其更快具有直接的实用价值。然而，先前的工作主要集中于优化大型系统中的某些部分，如内核或内存。在本工作中，我们采取整体方法，应用智能体自进化来端到端地优化整个系统。我们的SEIS（自进化推理系统）通过具有继承经验和代码更改的迭代会话，在无需人工干预的情况下自主优化整个mini-sglang引擎。在H100上服务Qwen3-0.6B时，所得引擎的吞吐量达到原始mini-sglang实现的3.27倍，并在单请求工作负载中超越了vLLM、TensorRT-LLM和SGLang等最先进引擎。SEIS优化的推理引擎的正确性通过数值差异和数学及长上下文检索任务的下游准确性进行测试。代码和会话历史表明，加速来自对整个引擎的重新设计，且基于早期会话的构建优于独立尝试。这些结果表明，智能体自进化可以端到端地优化复杂系统。评估也必须随引擎一起进化，让智能体进化评估是自然的下一步。
英文摘要
Inference systems determine how fast and how cheaply language models can be served, so making them faster has direct practical value. However, prior work focuses mostly on optimizing certain parts such as kernels or memory within the large system. In this work, we take a holistic approach and apply agentic self-evolution to optimize the whole system end-to-end. Our SEIS (Self-Evolving Inference Systems) autonomously optimizes the entire mini-sglang engine without human intervention through iterative sessions with inherited experiences and code changes. Serving Qwen3-0.6B on H100, the resulting engine reaches 3.27X the throughput of the original mini-sglang implementation and beats SOTA engines like vLLM, TensorRT-LLM, and SGLang in the single-request workload. The correctness of the optimized inference engine by SEIS is tested in terms of numerical difference and downstream accuracy on math and long-context retrieval tasks. The code and session histories show that the speedup comes from redesigning the whole engine and that building on earlier sessions beats independent attempts. These results suggest that agentic self-evolution can optimize a complex system end-to-end. The evaluation also has to evolve with the engine, and letting agents evolve it is a natural next step.
Comments21 pages
