迈向高效代理:推理架构与系统的一体化设计
Towards Efficient Agents: A Co-Design of Inference Architecture and System
AI总结:
本文提出AgentInfer框架,通过整合推理架构与系统设计,提升代理的效率与稳定性,减少无效token消耗,实现速度提升。
摘要:
大规模语言模型(LLM)基代理的快速发展解锁了自主多轮推理和工具增强决策的新可能性。然而,其在现实中的部署受到严重效率低下问题的阻碍,这些效率问题并非源于孤立的模型推理,而是源于推理循环、上下文增长和异构工具交互中累积的系统延迟。本文提出了AgentInfer,一个端到端代理加速的统一框架,弥合了推理优化与架构设计之间的鸿沟。我们将问题分解为四个协同组件:AgentCollab,一种分层双模型推理框架,通过动态角色分配平衡大模型和小模型的使用;AgentSched,一种缓存感知的混合调度器,在异构请求模式下最小化延迟;AgentSAM,一种基于后缀自动机的推测解码方法,通过重用多会话语义内存实现低开销的推理加速;以及AgentCompress,一种语义压缩机制,能够异步地提炼和重组代理内存,而不会中断正在进行的推理。这些模块共同构成了一个自我进化引擎,能够在长时间推理任务中持续保持效率和认知稳定性。在BrowseComp-zh和DeepDiver基准测试中,通过这些方法的协同合作,AgentInfer将无效的token消耗减少了超过50%,在保持准确性的同时实现了整体1.8至2.5倍的速度提升。这些结果表明,优化代理任务完成而非仅仅追求每token吞吐量是构建可扩展、高效和自我改进智能系统的关键。
英文摘要:
The rapid development of large language model (LLM)-based agents has unlocked new possibilities for autonomous multi-turn reasoning and tool-augmented decision-making. However, their real-world deployment is hindered by severe inefficiencies that arise not from isolated model inference, but from the systemic latency accumulated across reasoning loops, context growth, and heterogeneous tool interactions. This paper presents AgentInfer, a unified framework for end-to-end agent acceleration that bridges inference optimization and architectural design. We decompose the problem into four synergistic components: AgentCollab, a hierarchical dual-model reasoning framework that balances large- and small-model usage through dynamic role assignment; AgentSched, a cache-aware hybrid scheduler that minimizes latency under heterogeneous request patterns; AgentSAM, a suffix-automaton-based speculative decoding method that reuses multi-session semantic memory to achieve low-overhead inference acceleration; and AgentCompress, a semantic compression mechanism that asynchronously distills and reorganizes agent memory without disrupting ongoing reasoning. Together, these modules form a Self-Evolution Engine capable of sustaining efficiency and cognitive stability throughout long-horizon reasoning tasks. Experiments on the BrowseComp-zh and DeepDiver benchmarks demonstrate that through the synergistic collaboration of these methods, AgentInfer reduces ineffective token consumption by over 50%, achieving an overall 1.8-2.5 times speedup with preserved accuracy. These results underscore that optimizing for agentic task completion-rather than merely per-token throughput-is the key to building scalable, efficient, and self-improving intelligent systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。