工具过度使用错觉：为什么大语言模型更倾向于使用外部工具而非内部知识？
The Tool-Overuse Illusion: Why Does LLM Prefer External Tools over Internal Knowledge?
AI总结：
本文揭示大语言模型在推理中普遍存在工具过度使用现象，通过分析内部知识可用性区域，提出知识感知边界对齐策略，减少工具使用82.8%，并发现仅以结果奖励会鼓励工具过度使用，通过平衡奖励信号可减少66.7%的无用工具调用。
摘要：
配备外部工具能有效弥补大语言模型内部推理的不足。然而，它引入了一个关键但尚未深入探讨的现象：工具过度使用，即推理过程中不必要的工具使用。本文首先揭示这一现象在不同大语言模型中普遍存在。我们通过两个关键视角实验性地阐明其内在机制：（1）通过分析不同内部知识可用性区域的工具使用行为，我们识别出一种\textit{知识认知错觉}：模型错误判断内部知识边界，无法准确感知其实际知识可用性。为缓解这一问题，我们提出基于直接偏好优化的知识感知边界对齐策略，使工具使用减少82.8%，同时提升准确性。 （2）其次，我们通过可视化增强工具的训练过程，建立了奖励结构与工具使用行为之间的因果联系。它揭示出\textit{仅以结果奖励}会无意间鼓励工具过度使用，因为只奖励最终正确性，而不考虑工具效率。为验证这一点，我们在训练过程中平衡奖励信号，而非依赖仅以结果奖励，从而减少不必要的工具调用，分别减少了66.7%（7B）和60.7%（32B），而无需牺牲准确性。最后，我们通过这两个视角提供了理论依据以理解工具过度使用。
英文摘要：
Equipping LLMs with external tools effectively addresses internal reasoning limitations. However, it introduces a critical yet under-explored phenomenon: tool overuse, the unnecessary tool-use during reasoning. In this paper, we first reveal this phenomenon is pervasive across diverse LLMs. We then experimentally elucidate its underlying mechanisms through two key lenses: (1) First, by analyzing tool-use behavior across different internal knowledge availability regions, we identify a \textit{knowledge epistemic illusion}: models misjudge internal knowledge boundaries and fail to accurately perceive their actual knowledge availability. To mitigate this, we propose a knowledge-aware epistemic boundary alignment strategy based on direct preference optimization, which reduces tool usage in by 82.8\% while yielding an accuracy improvement. (2) Second, we establish a causal link between reward structures and tool-use behavior by visualizing the tool-augmented training process. It reveals that \textit{outcome-only rewards} inadvertently encourage tool overuse by rewarding only final correctness, regardless of tool efficiency. To verify this, we balance reward signals during training rather than relying on outcome-only rewards, cutting unnecessary tool calls by 66.7\% (7B) and 60.7\% (32B) without sacrificing accuracy. Finally, we provide theoretical justification in this two lenses to understand tool overuse.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
