“According to ……”：提示语言模型可改进对预训练数据的引用
"According to ...": Prompting Language Models Improves Quoting from Pre-Training Data
AI总结：
本文提出 according-to prompting 和 QUIP-Score，引导并量化 LLM 对预训练语料的引用，实验证明该方法可提升生成依据与任务表现，且模型能按要求调节依据程度。
摘要：
大型语言模型（LLM）尽管在事实数据上进行过预训练，仍可能产生幻觉并生成虚假信息。受新闻写作中“据消息人士称”这一手法启发，我们提出 according-to prompting（“根据……”提示）：引导 LLM 使回答基于先前观察到的文本。为量化这种 grounding（依据锚定），我们提出一种新的评估指标 QUIP-Score，用于衡量模型生成的答案能够在底层文本语料中直接找到的程度。我们在 Wikipedia、PubMed 和美国法律税法三个语料上进行实验，结果表明这些提示在我们的指标下提升了依据锚定，并且还常带来改进终端任务性能的额外收益。此外，要求模型降低依据锚定（或依据其他语料进行锚定）的提示确实会降低 QUIP-Score，这表明 LLM 能够根据请求增加或减少有依据的生成。
英文摘要：
Large Language Models (LLMs) may hallucinate and generate fake information, despite pre-training on factual data. Inspired by the journalistic device of "according to sources", we propose according-to prompting: directing LLMs to ground responses against previously observed text. To quantify this grounding, we propose a novel evaluation metric (QUIP-Score) that measures the extent to which model-produced answers are directly found in underlying text corpora. We illustrate with experiments on three corpora (Wikipedia, PubMed, and the U.S. legal tax code) that these prompts improve grounding under our metrics, with the additional benefit of often improving end-task performance. Furthermore, prompts that ask the model to decrease grounding (or to ground to other corpora) indeed decrease QUIP-Score, indicating the ability of LLMs to increase or decrease grounded generations on request.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
