探究提示与外部工具在大语言模型幻觉率中的作用
Investigating the Role of Prompting and External Tools in Hallucination Rates of Large Language Models
AI总结：
本文实证评估多种提示策略及工具调用智能体对大语言模型幻觉率的影响，发现最佳提示方法因问题而异，简单方法常更有效，而外部工具会显著增加幻觉。
摘要：
大语言模型（LLMs）是在大量人类可读文本语料上训练而成的强大计算模型，使其能够执行通用语言理解与生成任务。由于其在各类自然语言处理（NLP）任务上的卓越性能，LLMs 已在工业界和学术界获得显著关注。尽管取得了这些成功，LLMs 仍经常产生不准确内容，通常被称为幻觉。提示工程是指为 LLM 设计和构建指令以执行特定任务的过程，已成为缓解幻觉的一种关键方法。本文对旨在减少 LLM 幻觉的不同提示策略和框架进行了全面的实证评估。研究将多种提示技术应用于广泛的基准数据集，以评估每种方法的准确率和幻觉率。此外，本文还在相同基准上考察了工具调用智能体（即通过外部工具增强、从而获得语言生成之外能力的 LLM）对幻觉率的影响。研究结果表明，最佳提示技术取决于问题类型，并且在减少幻觉方面，较简单的技术往往优于更复杂的方法。此外，研究还表明，由于使用外部工具带来额外复杂性，LLM 智能体可能表现出显著更高的幻觉率。
英文摘要：
Large Language Models (LLMs) are powerful computational models trained on extensive corpora of human-readable text, enabling them to perform general-purpose language understanding and generation. LLMs have garnered significant attention in both industry and academia due to their exceptional performance across various natural language processing (NLP) tasks. Despite these successes, LLMs often produce inaccuracies, commonly referred to as hallucinations. Prompt engineering, the process of designing and formulating instructions for LLMs to perform specific tasks, has emerged as a key approach to mitigating hallucinations. This paper provides a comprehensive empirical evaluation of different prompting strategies and frameworks aimed at reducing hallucinations in LLMs. Various prompting techniques are applied to a broad set of benchmark datasets to assess the accuracy and hallucination rate of each method. Additionally, the paper investigates the influence of tool-calling agents (LLMs augmented with external tools to enhance their capabilities beyond language generation) on hallucination rates in the same benchmarks. The findings demonstrate that the optimal prompting technique depends on the type of problem, and that simpler techniques often outperform more complex methods in reducing hallucinations. Furthermore, it is shown that LLM agents can exhibit significantly higher hallucination rates due to the added complexity of external tool usage.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
