长视界LLM智能体服务的并行上下文压缩
Parallel Context Compaction for Long-Horizon LLM Agent Serving
AI总结：
针对长视界LLM智能体对话历史超出上下文窗口的问题，提出并行压缩方法，通过细粒度控制摘要体积和提示工程，在HotpotQA和LoCoMo基准上相比顺序基线降低端到端耗时并提升压缩吞吐量。
摘要：
长视界LLM智能体积累的对话历史会逐渐增长，最终超出模型的上下文窗口。基于LLM摘要的上下文压缩可以保持对话有界，但摘要本质上有损，且阻塞调用会暂停智能体推理数十秒。此外，由于提示指令基本被忽略，操作者无法细粒度控制摘要体积，随着上下文增长，模型生成的输出令牌数量及其保留的信息在不同运行间波动显著，使得智能体保留的知识在不同运行间不可预测。我们引入了长视界智能体流的 extbf{并行压缩}，并在HotpotQA多跳问答和LoCoMo长上下文对话基准上，针对8B到120B参数的四个骨干模型（混合密集和MoE架构，包括推理和非推理模型）与顺序同步基线进行对比。并行压缩使操作者能够细粒度、可预测地控制摘要体积，并支持每块更针对性的提示工程。在匹配的压缩解码体积下，它相比顺序基线减少了端到端耗时并提升了压缩吞吐量。
英文摘要：
Long-horizon LLM agents accumulate growing conversation histories that eventually exceed the model's context window. Context compaction via LLM-based summarization keeps the conversation bounded, but summarization is inherently lossy and the blocking call stalls agent inference for tens of seconds. Moreover, the operator has no fine-grained control over summary volume since prompt instructions are largely ignored, and as context grows, both the amount of output tokens the model produces and the information it retains fluctuate substantially from run to run, making the agent's retained knowledge unpredictable across runs. We introduce \textbf{parallel compaction} for long-horizon agentic flows and characterize it against the sequential synchronous baseline across four backbones spanning 8B to 120B parameters, mixing dense and MoE architectures with reasoning and non-reasoning models, on the HotpotQA multi-hop QA and LoCoMo long-context dialogue benchmarks. Parallel compaction gives the operator fine-grained, predictable control over summary volume and enables more targeted prompt engineering per block. At matched compaction decode volume, it reduces end-to-end wall time and improves compaction throughput over the sequential baseline.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
