语言模型逻辑推理能力的改进：基于可微符号编程
Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming
AI总结：
本文提出DSR-LM可微符号推理框架，结合预训练语言模型与符号模块，学习加权规则并应用语义损失，在演绎推理基准上准确率提升超20%，且优于多种基线。
摘要：
预训练的大语言模型（LMs）尽管在规模和组合性方面取得了进步，但在可靠地进行逻辑推理方面仍存在困难。在这项工作中，我们通过符号编程的视角来应对这一挑战。我们提出了DSR-LM，一个可微符号推理框架，其中预训练的语言模型负责感知事实知识，而符号模块则执行演绎推理。与依赖手工制作逻辑规则的工作不同，我们的可微符号推理框架能够高效地学习加权规则，并应用语义损失来进一步改进语言模型。DSR-LM具有可扩展性、可解释性，并允许轻松整合先验知识，从而支持广泛的符号编程，以稳健地推导出逻辑结论。我们的实验结果表明，DSR-LM提高了预训练语言模型的逻辑推理能力，在演绎推理基准测试上准确率显著提高了超过20%。此外，在面对序列长度的系统性变化时，DSR-LM优于多种竞争性基线方法。
英文摘要：
Pre-trained large language models (LMs) struggle to perform logical reasoning reliably despite advances in scale and compositionality. In this work, we tackle this challenge through the lens of symbolic programming. We propose DSR-LM, a Differentiable Symbolic Reasoning framework where pre-trained LMs govern the perception of factual knowledge, and a symbolic module performs deductive reasoning. In contrast to works that rely on hand-crafted logic rules, our differentiable symbolic reasoning framework efficiently learns weighted rules and applies semantic loss to further improve LMs. DSR-LM is scalable, interpretable, and allows easy integration of prior knowledge, thereby supporting extensive symbolic programming to robustly derive a logical conclusion. The results of our experiments suggest that DSR-LM improves the logical reasoning abilities of pre-trained language models, resulting in a significant increase in accuracy of over 20% on deductive reasoning benchmarks. Furthermore, DSR-LM outperforms a variety of competitive baselines when faced with systematic changes in sequence length.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
