arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.10380cs.AI
收藏

Agent-X:边缘设备上AI代理的全流水线加速

Agent-X: Full Pipeline Acceleration of On-device AI Agents

KAIST(韩国科学技术院)

Jinha Chung, Byeongjun Shin, Jiin Kim, Minsoo Rhu

AI总结:

本文提出Agent-X框架,通过优化预填充和解码阶段,实现边缘设备上AI代理的加速,无精度损失且集成简便,首次系统性消除代理延迟瓶颈。

摘要:

基于LLM的代理在任务上表现优异,但在边缘设备上存在较高的端到端延迟。我们引入Agent-X,一个软件-only、精度保持的框架,加速边缘设备上代理的工作负载的预填充和解码阶段。Agent-X的两个关键组件通过重写提示来利用针对代理特定输入令牌模式的前缀缓存,并启用无需LLM的推测解码以快速生成令牌并最小化开销。在代表性代理工作负载上,Agent-X在实际系统中实现了1.61倍的端到端加速,无精度损失,并可无缝集成到现有边缘AI代理中。据我们所知,这是首次系统性地表征并消除边缘代理延迟瓶颈的工作。

英文摘要:

LLM-based agents deliver state-of-the-art performance across tasks but incur high end-to-end latency on edge devices. We introduce Agent-X, a software-only, accuracy-preserving framework that accelerates both the prefill and decode stages of on-device agent workloads. Agent-X's two key components rewrite prompts to leverage prefix caching tailored to agent-specific input-token patterns and enable LLM-free speculative decoding for fast token generation with minimal overhead. On representative agentic workloads, Agent-X achieves a 1.61x end-to-end speedup in real systems with no accuracy loss and can be seamlessly integrated into existing on-device AI agents. To the best of our knowledge, ours is the first to systematically characterize and eliminate latency bottlenecks in on-device agents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑