思维逻辑:将逻辑注入上下文以实现大语言模型的完全推理
Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Models
AI总结:
本文提出 Logic-of-Thought 提示方法,通过命题逻辑生成扩展逻辑信息并注入原始上下文,以弥补逻辑表达式提取中的信息遗漏,从而显著提升大语言模型在多个逻辑推理任务上的表现。
摘要:
大语言模型(LLMs)在各种任务中展现了卓越的能力,但它们在复杂逻辑推理任务上的表现仍不令人满意。尽管一些提示方法(如思维链 Chain-of-Thought)在一定程度上可以提高 LLMs 的推理能力,但它们存在不忠实的问题,即推导出的结论可能与生成的推理链不一致。为了解决这一问题,一些研究采用命题逻辑的方法来进一步增强 LLMs 的逻辑推理能力。然而,这些方法在提取逻辑表达式时可能存在的遗漏会导致逻辑推理过程中的信息丢失,从而产生错误结果。为此,我们提出了 Logic-of-Thought(LoT)提示方法,该方法利用命题逻辑生成扩展的逻辑信息描述,并将其作为对原始上下文的额外增强,从而确保信息完整性并提高逻辑推理能力。LoT 与现有提示方法正交,可以无缝集成。大量实验表明,LoT 在五个逻辑推理任务上以显著幅度提升了各种提示方法的性能。特别是,LoT 将 Chain-of-Thought 在 ReClor 数据集上的性能提高了 +4.35%,将带自洽性的 Chain-of-Thought(Chain-of-Thought with Self-Consistency)在 RuleTaker 数据集上的性能提高了 +3.52%,并将 Tree-of-Thoughts 在 ProofWriter 数据集上的性能提高了 +8%。
英文摘要:
Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks but their performance in complex logical reasoning tasks remains unsatisfactory. Although some prompting methods, such as Chain-of-Thought, can improve the reasoning ability of LLMs to some extent, they suffer from an unfaithful issue where derived conclusions may not align with the generated reasoning chain. To address this issue, some studies employ the approach of propositional logic to further enhance logical reasoning abilities of LLMs. However, the potential omissions in the extraction of logical expressions in these methods can cause information loss in the logical reasoning process, thereby generating incorrect results. To this end, we propose Logic-of-Thought (LoT) prompting which employs propositional logic to generate expanded logical information descriptions and utilizes them as an additional augmentation to original contexts, thereby ensuring information completeness and enhancing logical reasoning ability. LoT is orthogonal to existing prompting methods and can be seamlessly integrated with them. Extensive experiments demonstrate that LoT boosts the performance of various prompting methods with a striking margin across five logical reasoning tasks. In particular, LoT enhances Chain-of-Thought's performance on the ReClor dataset by +4.35%, improves Chain-of-Thought with Self-Consistency's performance on the RuleTaker dataset by +3.52%, and boosts performance of Tree-of-Thoughts on the ProofWriter dataset by +8%.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。