arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.16682cs.DCcs.AI
收藏

KAIROS:具有状态的、基于上下文的节能代理推理服务

KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving

University of Illinois, Urbana Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Michigan(密歇根大学)

Yichao Yuan, Mosharaf Chowdhury, Nishil Talati

AI总结:

本文提出KAIROS,一种基于上下文的节能代理AI服务系统,通过管理GPU频率、实例并发性和请求分配,实现27%的平均能耗降低,同时保持性能。

摘要:

随着代理AI成为主要的工作负载类别,电力已成为AI推理的核心瓶颈。本文分析发现,代理服务的行为与单轮LLM服务有根本差异:每个请求携带长期上下文,且在工具交织的回合中持续演变。降低GPU频率可能导致系统进入thrashing状态,使内存压力显著增加性能和能耗。KAIROS利用代理上下文作为第一控制信号,联合管理GPU频率、实例并发性和多实例请求分配,从而在内存余量存在时节省电力,同时避免thrashing并保持性能目标。KAIROS通过跟踪请求、适应上下文增长和代理进展,并在实例间路由代理,共同提升能耗效率和内存稳定性。在多样化的软件和数据工程代理任务中,KAIROS实现了27%(最高39.8%)的平均能耗降低,同时满足性能目标。

英文摘要:

Power has become a central bottleneck for AI inference. This problem is becoming more urgent as agentic AI emerges as a major workload class, yet prior power-management techniques focus almost entirely on single-turn LLM serving. Our analysis shows that agentic serving behaves fundamentally differently: each request carries long-lived context that evolves across tool-interleaved turns, and lowering GPU frequency can push the system into a thrashing regime where memory pressure sharply worsens both performance and power efficiency. These observations show that power optimization for agentic serving requires rethinking. We present KAIROS, a context-aware power optimization system for agentic AI serving. KAIROS uses agent context as a first-class control signal to jointly manage GPU frequency, per-instance concurrency, and multi-instance request placement. This enables KAIROS to save power when memory headroom exists while avoiding thrashing and preserving performance targets. At a high level, KAIROS tracks requests at agent granularity, adapts local control to context growth and agent progress, and routes agents across instances to jointly improve power efficiency and memory stability. Evaluated across diverse software and data engineering agentic tasks, KAIROS achieves an average of 27% (up to 39.8%) power reduction while meeting the performance targets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑