大型语言模型是零示例学习者
Large Language Models are Null-Shot Learners
AI总结：
本文提出null-shot prompting，通过引导LLMs利用不存在的“Examples”信息来利用幻觉，在多数数据集上提升性能，并可作为检测LLMs幻觉程度的工具。
摘要：
本文提出了null-shot prompting。该方法通过指示大型语言模型（LLMs）利用所提供上下文中从未存在过的“Examples”部分的信息来执行任务，从而利用LLMs中的幻觉现象。尽管减少幻觉对于LLMs的日常和关键应用至关重要且不可忽视，我们提出，在当前这些LLMs仍然会产生幻觉的背景下，实际上可以通过利用幻觉来提升任务执行的性能，相较于标准的zero-shot prompting。在八个LLMs上的实验表明，在八个数据集（包括阅读理解、算术推理和闭卷问答）的大多数中都观察到了性能提升。在不同LLMs中观察到的相对性能提升的不一致性，也潜在地表明每个模型具有不同程度的固有幻觉。这些差异表明，可以利用null-shot prompting作为一种方法，通过现有的基准数据集来检测LLMs中的幻觉程度。我们还进行了消融研究，包括对结合了zero-shot chain-of-thought prompting思想的修改版null-shot prompting进行实验，结果显示出不同的结果趋势。
英文摘要：
This paper presents null-shot prompting. Null-shot prompting exploits hallucination in large language models (LLMs) by instructing LLMs to utilize information from the "Examples" section that never exists within the provided context to perform a task. While reducing hallucination is crucial and non-negligible for daily and critical uses of LLMs, we propose that in the current landscape in which these LLMs still hallucinate, it is possible, in fact, to exploit hallucination to increase performance in performing tasks compared to standard zero-shot prompting. Experiments with eight LLMs show improvements in performance across the majority of eight datasets, including reading comprehension, arithmetic reasoning, and closed-book question answering. The observed inconsistency in increased relative performance across the LLMs also potentially indicates a different degree of inherent hallucination in each model. These differences show that it is possible to utilize null-shot prompting as a way to detect degrees of hallucination in LLMs using existing benchmarking datasets. We also perform ablation studies, including experimenting with a modified version of null-shot prompting that incorporates ideas from zero-shot chain-of-thought prompting, which shows different trends of results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
