AI 中文总结
针对多租户LLM前缀缓存的准入责任缺口，提出PrefixShield方法，在vLLM中实现后可显著提升受害者缓存命中率，恢复良性行为并优化资源利用。
AI 中文摘要
共享前缀缓存将图形处理器（GPU）内存转化为跨大语言模型（LLM）租户共享的持久状态。实现新键值（KV）块的组可能迫使其他组丢失可复用状态，但请求时调度器仅考虑瞬态服务，替换策略主要依据对象价值排序，静态分区则会浪费空闲容量，我们将这种不匹配称为准入责任缺口。为解决该问题，我们提出PrefixShield，该方法对新实现的完整KV块进行计量，在请求间传递责任，在仍存在债务时管控复用升级，并利用预计债务选择提供驱逐候选的组。我们在vLLM中实现了PrefixShield。在单次污染的配对运行中，PrefixShield相比最近最少使用（LRU）策略将受害者缓存命中率提高9.39个百分点，相比S3-FIFO提高8.64个百分点，在4096块规模下将受害者命中率从4.92%恢复至84.87%；在两次重放场景下，相比S3-FIFO提高2.00个百分点，同时保留良性ShareGPT行为和对空闲容量的工作保持访问权限，在债务存在时延迟重放可获得35.16个百分点的优势。这些结果表明，对象价值信号决定保留对象，而持久责任决定哪个组承担回收压力。
英文摘要
Shared prefix caching turns Graphics Processing Unit (GPU) memory into persistent state shared across Large Language Model (LLM) tenants. A group that materializes new Key-Value (KV) blocks can force another to lose reusable state, yet request-time schedulers account for transient service, replacement policies primarily rank object value, and static partitioning strands idle capacity. We call this mismatch the admission-responsibility gap. To close it, we propose PrefixShield, which meters newly materialized full KV blocks, carries responsibility across requests, gates reuse promotion while debt remains, and uses projected debt to select the group supplying eviction candidates. We implement PrefixShield in vLLM. In paired runs under one-touch pollution, PrefixShield improves victim cache hit ratio by 9.39 percentage points over the Least Recently Used (LRU) policy and 8.64 points over S3-FIFO, restoring the victim from 4.92% to 84.87% at 4096-block scale, and gains 2.00 points over S3-FIFO under two-pass replay. It preserves benign ShareGPT behavior and work-conserving access to idle capacity. Delayed replay yields a 35.16-point advantage while debt remains. These results show that object-value signals rank what to retain, while persistent responsibility determines which group bears reclamation pressure.
