不要破坏缓存：对长周期智能体任务中提示缓存的评估
Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks
AI总结：
本文评估了提示缓存对长周期智能体任务的影响，发现通过策略性缓存控制可显著降低API成本并提升响应速度。
摘要：
近年来，大型语言模型（LLM）智能体的进步使得需要大量工具调用的复杂多轮智能体任务成为可能，其中对话可以跨越数十次API调用，且上下文窗口越来越大。然而，尽管主要的LLM提供商提供了提示缓存以减少成本和延迟，但其对智能体工作负载的好处在研究文献中仍处于探索阶段。据我们所知，没有先前的工作量化这些成本节省或比较多轮智能体任务中的缓存策略。我们对三个主要的LLM提供商（OpenAI、Anthropic和Google）的提示缓存进行了全面评估，并比较了三种缓存策略，包括完整的上下文缓存、仅系统提示的缓存以及排除动态工具结果的缓存。我们在DeepResearch Bench上进行了评估，这是一个多轮智能体基准测试，其中智能体自主执行现实世界的网络搜索工具调用来回答复杂的研究问题，测量API成本和首次令牌时间（TTFT）超过500次智能体会话，使用10,000个令牌的系统提示。我们的结果表明，提示缓存将API成本减少了41-80%，并在各提供商中提高了首次令牌时间13-31%。我们发现，战略性提示缓存块控制，如将动态内容放在系统提示的末尾、避免动态传统功能调用以及排除动态工具结果，比简单的完整上下文缓存提供更一致的好处，后者可能会反常地增加延迟。对提示大小（500-50,000个令牌）和工具调用次数（3-50次）的消融研究证明了普遍的线性成本和TTFT收益，在提供商缓存令牌最小值之后，并揭示了不同变体中的提供商特定策略差异。我们为在生产智能体系统中实现提示缓存提供了细致的讨论和指导。
英文摘要：
Recent advancements in Large Language Model (LLM) agents have enabled complex multi-turn agentic tasks requiring extensive tool calling, where conversations can span dozens of API calls with increasingly large context windows. However, although major LLM providers offer prompt caching to reduce cost and latency, its benefits for agentic workloads remain underexplored in the research literature. To our knowledge, no prior work quantifies these cost savings or compares caching strategies for multi-turn agentic tasks. We present a comprehensive evaluation of prompt caching across three major LLM providers (OpenAI, Anthropic, and Google) and compare three caching strategies, including full context caching, system prompt only caching, and caching that excludes dynamic tool results. We evaluate on DeepResearch Bench, a multi-turn agentic benchmark where agents autonomously execute real-world web search tool calls to answer complex research questions, measuring both API cost and time to first token (TTFT) across over 500 agent sessions with 10,000-token system prompts. Our results demonstrate that prompt caching reduces API costs by 41-80% and improves time to first token by 13-31% across providers. We find that strategic prompt cache block control, such as placing dynamic content at the end of the system prompt, avoiding dynamic traditional function calling, and excluding dynamic tool results, provides more consistent benefits than naive full-context caching, which can paradoxically increase latency. An ablation study across prompt sizes (500-50,000 tokens) and tool call counts (3-50) demonstrates universal linear cost and TTFT benefits, after the provider caching token minimum, and reveal provider-specific strategy discrepancies across variants. We provide nuanced discussion and guidance for implementing prompt caching in production agentic systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
