DOCSCHISEL：面向大语言模型智能体的自适应工具文档优化框架
DOCSCHISEL: Adaptive Tool Documentation Optimization Framework for LLM Agents
浏览论文内容
中文总结 AI 辅助
该研究针对LLM智能体工具文档的异质性与泛化问题，提出DocsChisel自适应优化框架，经实验较原始文档及EasyTool、DRAFT基线大幅提升任务成功率，且开销有限。
中文摘要 AI 辅助
大语言模型（LLM）越来越依赖外部工具来完成复杂的现实任务，这使得工具文档成为LLM智能体的关键基础资源。现有研究主要聚焦于提升LLM智能体的工具使用能力，而大多将工具文档视为固定输入。尽管近期有若干工作尝试通过重写或压缩来优化工具文档，但目前人们对工具文档包含的信息如何在不同场景下影响智能体性能知之甚少。为填补这一空白，我们针对LLM智能体的工具文档开展了大规模实证研究。我们的研究揭示，现有工具文档提供的信息领域存在显著异质性；此外，不同信息领域的有效性高度依赖于任务领域、LLM主干模型和智能体范式，这表明不存在固定的工具文档能在多样的智能体场景中持续泛化。基于这些发现，我们提出了DocsChisel，一种面向LLM智能体的自适应工具文档优化框架。DocsChisel分析目标LLM智能体的失败执行轨迹，以识别与文档相关的问题，并通过为每个工具添加、移除和优化信息领域来迭代优化工具文档。我们将DocsChisel与两个最先进的基线方法EasyTool和DRAFT进行了评估。实验结果显示，相较于原始工具文档，DocsChisel将LLM智能体的任务成功率提升了95.89%；相较于现有基线方法，平均提升了75.15%，同时仅产生有限的优化时间和令牌开销。
英文摘要
Large language models (LLMs) increasingly rely on external tools to accomplish complex real-world tasks, making tool documentation a critical grounding resource for LLM agents. Existing studies mainly focus on improving the tool-use capabilities of LLM agents, while largely treating tool documentation as a fixed input. Although several recent works attempt to optimize tool documentation through rewriting or compression, little is known about how the information contained in tool documentation affects agent performance across different settings. To bridge this gap, we conduct a large-scale empirical study on tool documentation for LLM agents. Our study reveals substantial heterogeneity in the information fields provided by existing tool documentation. Moreover, the effectiveness of different information fields is highly dependent on the task domain, LLM backbone, and agent paradigm, indicating that no fixed tool documentation can consistently generalize across diverse agent settings. Motivated by these findings, we propose DocsChisel, an adaptive tool documentation optimization framework for LLM agents. DocsChisel analyzes failed execution traces of a target LLM agent to identify documentation-related issues, and iteratively optimizes tool documentation by adding, removing, and refining information fields for each tool. We evaluate DocsChisel against two state-of-the-art baselines, i.e., EasyTool and DRAFT. Experimental results show that DocsChisel improves the task success rate of LLM agents by 95.89% over the original tool documentation and by 75.15%, on average, over existing baselines, while incurring limited optimization time and token overhead
