SparseEngine：稀疏优先推理引擎
SparseEngine: Sparse-First Inference Engine
AI总结：
SparseEngine是一个稀疏优先的推理引擎，通过共享生命周期契约和Chain Cache、Prefix-Cache剪枝，支持多种稀疏方法，实现高吞吐、快速解码，显著提升长上下文智能体性能。
AI中文摘要：
长上下文LLM智能体会累积交互历史，这给KV缓存内存和注意力计算带来压力。尽管稀疏注意力能降低这些成本，但异构的缓存表示和工作流阻碍了与现有推理引擎的集成，而先前的稀疏服务抽象仅支持特定的布局或工作流。我们提出了SparseEngine，一个从零构建的、稀疏优先的推理引擎，其共享生命周期契约让每种方法能够控制其KV表示和计算，同时与通用服务基础设施协调状态转换。SparseEngine支持四类共15种方法，并通过Chain Cache实现跨请求状态管理，该方法能从保留的历史中恢复KV淘汰方法，以及可控的Prefix-Cache剪枝，它从选定的历史区域移除KV，同时保持逻辑前缀匹配。在保持方法质量的同时，SparseEngine在KV淘汰下实现了超过10倍的吞吐量提升，在匹配并发下解码速度比vLLM快2.5倍以上，并在智能体基准测试中实现了超过2倍的端到端加速。代码可在该https URL获取。
英文摘要：
Long-context LLM agents accumulate interaction histories that strain KV-cache memory and attention computation. Although sparse attention reduces these costs, heterogeneous cache representations and workflows hinder integration with existing inference engines, while prior sparse-serving abstractions support only specific layouts or workflows. We present SparseEngine, a ground-up, sparse-first inference engine whose shared lifecycle contract lets each method control its KV representation and computation while coordinating state transitions with common serving infrastructure. SparseEngine supports 15 methods across four categories and enables cross-request state management through Chain Cache, which resumes KV-eviction methods from retained history, and controllable Prefix-Cache Pruning, which removes KV from selected history regions while preserving logical-prefix matching. While maintaining method quality, SparseEngine delivers over 10x higher throughput with KV eviction, over 2.5x faster decoding at matched concurrency than vLLM, and over 2x end-to-end speedup on agent benchmarks. The code is available at https://github.com/CURRENTF/SparseEngine.
