通过神经符号集成实现逻辑一致的语言模型
Logically Consistent Language Models via Neuro-Symbolic Integration
AI总结：
针对LLM易生成非事实信息且推理时自相矛盾的问题，提出基于神经符号推理的损失函数，使模型与外部事实和规则保持逻辑一致，提升自洽性，支持多约束组合，并增强对未见事实知识的系统外推能力。
摘要：
大型语言模型（LLMs）是自然语言理解和生成的有前景的途径。然而，当前的LLMs远非可靠：它们容易生成非事实信息，更关键的是，当被提示推理世界实体之间的关系时，容易自相矛盾。这些问题目前通过大规模微调或将推理委托给外部工具来解决。在这项工作中，我们寻求折中方案，并引入一种基于神经符号推理的损失函数，教导LLM与外部的成组事实和规则保持逻辑一致，并提高自洽性，即使LLM在有限的事实集合上微调。我们的方法还允许以有原则的方式轻松地同时组合多个逻辑约束，生成在所有约束上更一致的LLM，并在给定约束上优于几个基线。此外，我们的方法允许LLM更系统地外推到未见但在未见数据集中表示的语义相似的事实知识。
英文摘要：
Large language models (LLMs) are a promising venue for natural language understanding and generation. However, current LLMs are far from reliable: they are prone to generating non-factual information and, more crucially, to contradicting themselves when prompted to reason about relations between entities of the world. These problems are currently addressed with large scale fine-tuning or by delegating reasoning to external tools. In this work, we strive for a middle ground and introduce a loss based on neuro-symbolic reasoning that teaches an LLM to be logically consistent with an external set of facts and rules and improves self-consistency even when the LLM is fine-tuned on a limited set of facts. Our approach also allows to easily combine multiple logical constraints at once in a principled way, delivering LLMs that are more consistent w.r.t. all constraints and improve over several baselines w.r.t. a given constraint. Moreover, our method allows LLMs to extrapolate to unseen but semantically similar factual knowledge, represented in unseen datasets, more systematically.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
