arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.08877cs.AI
收藏

量化预算约束下代理LLM搜索中的准确性与成本影响

Quantifying the Accuracy and Cost Impact of Design Decisions in Budget-Constrained Agentic LLM Search

Kyle McCleary, James Ghawaly

AI总结:

本研究探讨了预算约束下代理LLM搜索中搜索深度、检索策略和完成预算对准确性和成本的影响,并提供了可重复的评估方法和实验结果。

摘要:

代理检索增强生成(RAG)系统结合了迭代搜索、规划提示和检索后端,但在实际部署中会对工具调用和完成令牌施加显式的预算限制。我们通过受控测量研究,探讨搜索深度、检索策略和完成预算如何在固定约束下影响准确性和成本。使用预算约束代理搜索(BCAS),一种模型无关的评估工具,它可以揭示剩余预算并控制工具使用。我们对六个LLM和三个问答基准进行了比较。在不同模型和数据集上,增加的搜索会提高准确性,直到达到一个较小的上限;混合词法和密集检索结合轻量级重排序在我们的消融网格中产生最大的平均收益;较大的完成预算在HotpotQA类型的合成任务中最有帮助。这些结果为配置预算化的代理检索流水线提供了实用指导,并附有可重复使用的提示和评估设置。

英文摘要:

Agentic Retrieval-Augmented Generation (RAG) systems combine iterative search, planning prompts, and retrieval backends, but deployed settings impose explicit budgets on tool calls and completion tokens. We present a controlled measurement study of how search depth, retrieval strategy, and completion budget affect accuracy and cost under fixed constraints. Using Budget-Constrained Agentic Search (BCAS), a model-agnostic evaluation harness that surfaces remaining budget and gates tool use, we run comparisons across six LLMs and three question-answering benchmarks. Across models and datasets, accuracy improves with additional searches up to a small cap, hybrid lexical and dense retrieval with lightweight re-ranking produces the largest average gains in our ablation grid, and larger completion budgets are most helpful on HotpotQA-style synthesis. These results provide practical guidance for configuring budgeted agentic retrieval pipelines and are accompanied by reproducible prompts and evaluation settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑