发表机构
Seoul National University; KAIST; University of Illinois, Urbana-Champaign（首尔大学; 韩国科学技术院; 伊利诺伊大学厄巴纳-香槟分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对智能体服务中KV缓存寿命差异，提出Lachesis寿命感知放置层，按寿命将缓存置于HBM或HBF，延长HBF寿命1.19-3.13倍，并超过五年保修期。
AI 中文摘要
大语言模型（LLM）服务日益被智能体工作负载所主导，其中智能体及其子智能体在多次请求中累积上下文作为KV缓存，消耗大量内存。高带宽闪存（HBF）是一种有前景的解决方案，以HBM级别的读取带宽提供数量级更大的容量，但其有限的写入耐久性是关键限制因素。我们的核心洞察是，KV缓存应根据其寿命跨HBM和HBF进行放置。将寿命较短的数据放置在HBM中，可使HBM吸收智能体运行中更多的写入，并减少发送至HBF的写入量。由于智能体服务中KV缓存的寿命由编排智能体的程序（即harness）决定，我们分析其行为并识别出寿命差异的三个维度：时间维度、结构维度和跨工作进程维度。基于这些观察，我们提出Lachesis，一个位于智能体harness与服务引擎之间的寿命感知KV缓存放置层。在写入时，它根据每个片段的寿命将其放置在HBM或HBF中，并在片段不再被读取时释放其块。在基于轨迹的模拟中，与HBM优先放置相比，Lachesis将HBF寿命延长了1.19-3.13倍，达到3.3-12.2设备年。即使在满负载下持续24x7运行且满足严格SLO的情况下，HBF在多智能体轨迹上仍超过其五年保修期。
英文摘要
Large language model (LLM) serving is increasingly dominated by agentic workloads, in which agents and their sub-agents accumulate context as KV cache across many requests, consuming substantial memory. High-bandwidth flash (HBF) is a promising solution, providing an order of magnitude greater capacity at HBM-class read bandwidth, but its finite write endurance is the key limiting factor. Our key insight is that KV cache should be placed across HBM and HBF by its lifetime. Placing shorter-lived data in HBM lets HBM absorb more of an agent run's writes and sends less of them to HBF. As the lifetime of KV cache in agentic serving is dictated by the harness, the program that orchestrates the agents, we analyze its behavior and identify three axes along which lifetime diverges, temporal, structural, and inter-worker. Guided by these observations, we present Lachesis, a lifetime-aware KV cache placement layer between the agent harness and the serving engine. At write time, it places each segment in HBM or HBF according to its lifetime, and frees its blocks once the segment is no longer read. In trace-driven simulation, Lachesis extends HBF lifetime by 1.19-3.13x over HBM-first placement, reaching 3.3-12.2 device-years. Even under continuous 24x7 operation at the full load a tight SLO admits, HBF outlasts its five-year warranty on the multi-agent trace.
