利用知识链提示增强语言模型推理
Boosting Language Models Reasoning with Chain-of-Knowledge Prompting
AI总结：
针对思维链提示中推理链易出错的问题，提出知识链提示生成结构化知识证据，并引入F^2-Verification方法评估推理可靠性，通过指出错误证据促使模型重新思考，有效提升了多类推理任务的性能。
摘要：
近期，思维链（CoT）提示在复杂推理任务上取得了成功，其旨在设计一个简单的提示（如“让我们一步步思考”）或带有精心设计理由的多个上下文示例，以引导大型语言模型（LLMs）生成中间推理步骤。然而，生成的理由常伴随错误，导致产生不真实且不忠实的推理链。为减轻这一脆弱性，我们提出了一种新颖的知识链（CoK）提示，旨在引导LLMs以结构三元组的形式生成显式的知识证据片段。这受启发于人类行为，即在回答复杂问题前，我们可以在脑海中绘制思维导图或知识导图作为推理证据。得益于CoK，我们额外引入了F^2-Verification方法，从事实性和忠实性两方面评估推理链的可靠性。对于不可靠的响应，可以指出错误证据以提示LLM重新思考。大量实验表明，我们的方法能进一步提升常识、事实、符号和算术推理任务的性能。
英文摘要：
Recently, Chain-of-Thought (CoT) prompting has delivered success on complex reasoning tasks, which aims at designing a simple prompt like ``Let's think step by step'' or multiple in-context exemplars with well-designed rationales to elicit Large Language Models (LLMs) to generate intermediate reasoning steps. However, the generated rationales often come with mistakes, making unfactual and unfaithful reasoning chains. To mitigate this brittleness, we propose a novel Chain-of-Knowledge (CoK) prompting, where we aim at eliciting LLMs to generate explicit pieces of knowledge evidence in the form of structure triple. This is inspired by our human behaviors, i.e., we can draw a mind map or knowledge map as the reasoning evidence in the brain before answering a complex question. Benefiting from CoK, we additionally introduce a F^2-Verification method to estimate the reliability of the reasoning chains in terms of factuality and faithfulness. For the unreliable response, the wrong evidence can be indicated to prompt the LLM to rethink. Extensive experiments demonstrate that our method can further improve the performance of commonsense, factual, symbolic, and arithmetic reasoning tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
