This work introduces Symbolic-Aided Chain-of-Thought (CoT), an improved approach to standard CoT, for logical reasoning in large language models (LLMs). The key idea is to integrate lightweight symbolic representations into few-shot prompts, structuring the inference steps with a consistent strategy to make reasoning patterns more explicit within a non-interactive reasoning process. By incorporating these symbolic structures, Symbolic-Aided CoT preserves the generalizability of standard prompting techniques while enhancing the transparency, interpretability, and analyzability of LLM logical reasoning. Extensive experiments on four well-known logical reasoning benchmarks -- ProofWriter, FOLIO, ProntoQA, and LogicalDeduction, which cover diverse reasoning tasks and scenarios -- demonstrate the effectiveness of the proposed approach, particularly in complex reasoning tasks that require navigating multiple constraints or rules. Notably, Symbolic-Aided CoT consistently improves LLMs' reasoning capabilities across various model sizes and significantly outperforms conventional CoT on three out of four datasets, ProofWriter, ProntoQA, and LogicalDeduction.
Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming
Carnegie Mellon University（卡内基梅隆大学）；University of Pennsylvania（宾夕法尼亚大学）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Petuum Inc.（Petuum公司）
Deep neural networks, empowered by pre-trained language models, have achieved remarkable results in natural language understanding (NLU) tasks. However, their performances can drastically deteriorate when logical reasoning is needed. This is because NLU in principle depends on not only analogical reasoning, which deep neural networks are good at, but also logical reasoning. According to the dual-process theory, analogical reasoning and logical reasoning are respectively carried out by System 1 and System 2 in the human brain. Inspired by the theory, we present a novel framework for NLU called …
Adaptive LLM-Symbolic Reasoning via Dynamic Logical Solver Composition
Idiap Research Institute（Idiap研究机构）；École Polytechnique Fédérale de Lausanne（联邦理工学院洛桑分校）；School of Computer Science, University of Sheffield（谢菲尔德大学计算机科学学院）；Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）；Cancer Biomarker Centre, CRUK Manchester Institute（癌症生物标志物中心，CRUK曼彻斯特研究所）
Enhancing Logical Reasoning in Language Models via Symbolically-Guided Monte Carlo Process Supervision
School of Computer Science, University of Sheffield（谢菲尔德大学计算机科学学院）；School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦大学玛丽女王学院电子工程与计算机科学学院）；The Alan Turing Institute（艾伦·图灵研究所）
Progress in pre-trained language models has led to a surge of impressive results on downstream tasks for natural language understanding. Recent work on probing pre-trained language models uncovered a wide range of linguistic properties encoded in their contextualized representations. However, it is unclear whether they encode semantic knowledge that is crucial to symbolic inference methods. We propose a methodology for probing linguistic information for logical inference in pre-trained language model representations. Our probing datasets cover a list of linguistic phenomena required by major s…
Large language models (LLMs) have recently demonstrated an impressive ability to perform arithmetic and symbolic reasoning tasks, when provided with a few examples at test time ("few-shot prompting"). Much of this success can be attributed to prompting methods such as "chain-of-thought'', which employ LLMs for both understanding the problem description by decomposing it into steps, as well as solving each step of the problem. While LLMs seem to be adept at this sort of step-by-step decomposition, LLMs often make logical and arithmetic mistakes in the solution part, even when the problem is dec…
Matrix as Plan: Structured Logical Reasoning with Feedback-Driven Replanning
Faculty of Arts and Sciences（艺术与科学学院）；Beijing Normal University（北京师范大学）；Institute of Artificial Intelligence and Future Networks（人工智能与未来网络研究所）；Engineering Research Center of Cloud-Edge Intelligent Collaboration on Big Data, Ministry of Education（教育部云-边智能协同大数据工程研究中心）
Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions
Idiap Research Institute（Idiap 研究所）；School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；School of Computer Science, University of Sheffield（谢菲尔德大学计算机科学学院）；Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）；National Biomarker Centre (NBC), CRUK Manchester Institute（英国癌症研究中心曼彻斯特研究所国家生物标志物中心）
HOLMES: Evaluating Higher-Order Logical Reasoning in LLMs
State Key Laboratory of Multimedia Information Processing, Peking University（北京大学多媒体信息处理国家重点实验室）；School of Computer Science, Peking University（北京大学计算机科学学院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；YiXin-AILab, YIXIN（YiXin-AILab，YIXIN）
Prompting large language models has enabled significant recent progress in multi-step reasoning over text. However, when applied to text generation from semi-structured data (e.g., graphs or tables), these methods typically suffer from low semantic coverage, hallucination, and logical inconsistency. We propose MURMUR, a neuro-symbolic modular approach to text generation from semi-structured data with multi-step reasoning. MURMUR is a best-first search method that generates reasoning paths using: (1) neural and symbolic modules with specific linguistic and logical skills, (2) a grammar whose pr…
Verification and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Proving
University of Manchester（曼彻斯特大学）；Idiap Research Institute（Idiap研究所）；National Biomarker Centre, CRUK-MI, University of Manchester（英国曼彻斯特大学英国癌症研究中心曼彻斯特研究所国家生物标志物中心）
Enhancing Ethical Explanations of Large Language Models through Iterative Symbolic Refinement
University of Manchester（曼彻斯特大学）；Idiap Research Institute（Idiap研究所）；National Biomarker Centre, CRUK-MI, University of Manchester（曼彻斯特大学英国癌症研究中心曼彻斯特研究所国家生物标志物中心）
State Key Lab of Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；University of Toronto（多伦多大学）；Birkbeck, University of London（伦敦大学伯贝克学院）
Rose-Hulman Institute of Technology（罗斯-霍曼理工学院）；Indiana University（印第安纳大学）
Deep learning (DL) based language models achieve high performance on various benchmarks for Natural Language Inference (NLI). And at this time, symbolic approaches to NLI are receiving less attention. Both approaches (symbolic and DL) have their advantages and weaknesses. However, currently, no method combines them in a system to solve the task of NLI. To merge symbolic and deep learning methods, we propose an inference framework called NeuralLog, which utilizes both a monotonicity-based logical inference engine and a neural network language model for phrase alignment. Our framework models the…
We present Logical Optimal Actions (LOA), an action decision architecture of reinforcement learning applications with a neuro-symbolic framework which is a combination of neural network and symbolic knowledge acquisition approach for natural language interaction games. The demonstration for LOA experiments consists of a web-based interactive platform for text-based games and visualization for acquired knowledge for improving interpretability for trained rules. This demonstration also provides a comparison module with other neuro-symbolic approaches as well as non-symbolic state-of-the-art agen…
Constraints-Guided Diffusion Reasoner for Neuro-Symbolic Learning
Fudan University（复旦大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Academy of Artificial Intelligence for Science（上海人工智能科学研究院）；INFTECH；University of Waterloo（滑铁卢大学）
Deep reinforcement learning (RL) methods often require many trials before convergence, and no direct interpretability of trained policies is provided. In order to achieve fast convergence and interpretability for the policy in RL, we propose a novel RL method for text-based games with a recent neuro-symbolic framework called Logical Neural Network, which can learn symbolic and interpretable rules in their differentiable network. The method is first to extract first-order logical facts from text observation and external word meaning network (ConceptNet), then train a policy in the network with …
