大型语言模型的上下文忠实提示
Context-faithful Prompting for Large Language Models
AI总结：
本文研究如何通过基于观点的提示和反事实演示两种无需额外训练的提示策略，提升大型语言模型在知识冲突和带弃权预测场景下的上下文忠实度，并在机器阅读理解和关系抽取任务上验证了其有效性。
摘要：
大型语言模型（LLMs）编码了关于世界事实的参数化知识，并在知识驱动的NLP任务中表现出色。然而，对参数化知识的依赖可能导致它们忽略上下文线索，在上下文敏感的NLP任务（如知识获取任务）中导致错误预测。本文旨在从两个方面评估和增强LLMs的上下文忠实度：知识冲突和带弃权（不执行）的预测。我们证明，使用精心设计的提示策略可以显著提高LLMs的忠实度。特别是，我们发现基于观点的提示和反事实演示是最有效的方法。基于观点的提示将上下文重新表述为叙述者的陈述并询问叙述者的观点，而反事实演示使用包含虚假事实的实例来提高知识冲突情况下的忠实度。这两种技术都不需要额外训练。我们在两个标准NLP任务（机器阅读理解和关系抽取）的三个数据集上进行了实验，结果表明对上下文的忠实度有显著提升。代码和数据发布在https://github.com/wzhouad/context-faithful-llm。
英文摘要：
Large language models (LLMs) encode parametric knowledge about world facts and have shown remarkable performance in knowledge-driven NLP tasks. However, their reliance on parametric knowledge may cause them to overlook contextual cues, leading to incorrect predictions in context-sensitive NLP tasks (e.g., knowledge acquisition tasks). In this paper, we seek to assess and enhance LLMs' contextual faithfulness in two aspects: knowledge conflict and prediction with abstention. We demonstrate that LLMs' faithfulness can be significantly improved using carefully designed prompting strategies. In particular, we identify opinion-based prompts and counterfactual demonstrations as the most effective methods. Opinion-based prompts reframe the context as a narrator's statement and inquire about the narrator's opinions, while counterfactual demonstrations use instances containing false facts to improve faithfulness in knowledge conflict situations. Neither technique requires additional training. We conduct experiments on three datasets of two standard NLP tasks, machine reading comprehension and relation extraction, and the results demonstrate significant improvement in faithfulness to contexts. Code and data are released at https://github.com/wzhouad/context-faithful-llm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
