工具文档 enables Large Language Models 的零样本工具使用
Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models
AI总结：
提出以工具文档替代示例实现LLMs零样本工具使用，在6项跨模态任务中验证其性能与少样本相当甚至更优，并展示文档驱动自动启用新应用的能力。
摘要：
如今，大型语言模型（LLMs）通过提供少量工具使用的示例来学习使用新工具。然而，示例难以获取，且若选错示例会导致不良的偏见使用。即便在示例可得的罕见情况下，也缺乏原则性的选择协议来确定提供多少及哪些示例。随着任务复杂化，选择搜索呈组合增长，不可避免地变得难以处理。我们的工作提供了示例的替代方案：工具文档。我们提倡使用工具文档（即针对单个工具使用的描述）而非示例。我们通过在视觉和语言模态的6项任务上的三个主要实证发现来支持这一主张。首先，在现有基准上，仅含工具文档的零样本提示足以引发正确的工具使用，其性能与少样本提示相当。其次，在一个新收集的包含数百个可用工具API的现实工具使用数据集上，我们表明工具文档的价值显著高于示例，零样本文档的性能显著优于无文档的少样本。第三，我们通过将刚发布的未见最先进模型作为工具应用于图像生成和视频跟踪，突显了工具文档的优势。最后，我们强调利用工具文档自动启用新应用的可能性：仅使用 GroundingDino、Stable Diffusion、XMem 和 SAM 的文档，LLMs 即可重新实现刚发布的 Grounded-SAM 和 Track Anything 模型的功能。
英文摘要：
Today, large language models (LLMs) are taught to use new tools by providing a few demonstrations of the tool's usage. Unfortunately, demonstrations are hard to acquire, and can result in undesirable biased usage if the wrong demonstration is chosen. Even in the rare scenario that demonstrations are readily available, there is no principled selection protocol to determine how many and which ones to provide. As tasks grow more complex, the selection search grows combinatorially and invariably becomes intractable. Our work provides an alternative to demonstrations: tool documentation. We advocate the use of tool documentation, descriptions for the individual tool usage, over demonstrations. We substantiate our claim through three main empirical findings on 6 tasks across both vision and language modalities. First, on existing benchmarks, zero-shot prompts with only tool documentation are sufficient for eliciting proper tool usage, achieving performance on par with few-shot prompts. Second, on a newly collected realistic tool-use dataset with hundreds of available tool APIs, we show that tool documentation is significantly more valuable than demonstrations, with zero-shot documentation significantly outperforming few-shot without documentation. Third, we highlight the benefits of tool documentations by tackling image generation and video tracking using just-released unseen state-of-the-art models as tools. Finally, we highlight the possibility of using tool documentation to automatically enable new applications: by using nothing more than the documentation of GroundingDino, Stable Diffusion, XMem, and SAM, LLMs can re-invent the functionalities of the just-released Grounded-SAM and Track Anything models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
