This work describes a methodology that combines logic-based systems and connectionist systems. Our approach uses finite truth-valued Łukasiewicz logic, wherein every connective can be defined by a neuron in an artificial network. This allowed the injection of first-order formulas into a network architecture, and also simplified symbolic rule extraction. For that we trained a neural networks using the Levenderg-Marquardt algorithm, where we restricted the knowledge dissemination in the network structure. This procedure reduces neural network plasticity without drastically damaging the learning performance, thus making the descriptive power of produced neural networks similar to the descriptive power of Łukasiewicz logic language and simplifying the translation between symbolic and connectionist structures. We used this method for reverse engineering truth table and in extraction of formulas from real data sets.
Università degli Studi di Urbino “Carlo Bo”（乌尔比诺“卡洛·博”大学）
Opaque models belonging to the machine learning world are ever more exploited in the most different application areas. These models, acting as black boxes (BB) from the human perspective, cannot be entirely trusted if the application is critical unless there exists a method to extract symbolic and human-readable knowledge out of them. In this paper we analyse a recurrent design adopted by symbolic knowledge extractors for BB regressors - that is, the creation of rules associated with hypercubic input space regions. We argue that this kind of partitioning may lead to suboptimal solutions when t…
State Key Lab of Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；University of Toronto（多伦多大学）；Birkbeck, University of London（伦敦大学伯贝克学院）
Mila - Québec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；McGill University（麦吉尔大学）；HEC Montréal（蒙特利尔高等商学院）；CIFAR AI Chair（加拿大高等研究院人工智能讲席）
Answering complex first-order logic (FOL) queries on knowledge graphs is a fundamental task for multi-hop reasoning. Traditional symbolic methods traverse a complete knowledge graph to extract the answers, which provides good interpretation for each step. Recent neural methods learn geometric embeddings for complex queries. These methods can generalize to incomplete knowledge graphs, but their reasoning process is hard to interpret. In this paper, we propose Graph Neural Network Query Executor (GNN-QE), a neural-symbolic model that enjoys the advantages of both worlds. GNN-QE decomposes a comp…
We propose neural-symbolic integration for abstract concept explanation and interactive learning. Neural-symbolic integration and explanation allow users and domain-experts to learn about the data-driven decision making process of large neural models. The models are queried using a symbolic logic language. Interaction with the user then confirms or rejects a revision of the neural model using logic-based constraints that can be distilled into the model architecture. The approach is illustrated using the Logic Tensor Network framework alongside Concept Activation Vectors and applied to a Convol…
Rensselaer Polytechnic Institute（伦斯勒理工学院）；IBM T.J. Watson Research Center（IBM托马斯·J·沃森研究中心）；IBM Research（IBM研究院）
Most existing Time series classification (TSC) models lack interpretability and are difficult to inspect. Interpretable machine learning models can aid in discovering patterns in data as well as give easy-to-understand insights to domain specialists. In this study, we present Neuro-Symbolic Time Series Classification (NSTSC), a neuro-symbolic model that leverages signal temporal logic (STL) and neural network (NN) to accomplish TSC tasks using multi-view data representation and expresses the model as a human-readable, interpretable formula. In NSTSC, each neuron is linked to a symbolic express…
NASP-T: A Fuzzy Neuro-Symbolic Transformer for Logic-Constrained Aviation Safety Report Classification
Faculty of Science and Technology (REALTEK)（科学与技术学院）；Norwegian University of Life Sciences (NMBU)（挪威生命科学大学）；Dresden International University (DIU)（德累斯顿国际大学）
Deep reinforcement learning (RL) methods often require many trials before convergence, and no direct interpretability of trained policies is provided. In order to achieve fast convergence and interpretability for the policy in RL, we propose a novel RL method for text-based games with a recent neuro-symbolic framework called Logical Neural Network, which can learn symbolic and interpretable rules in their differentiable network. The method is first to extract first-order logical facts from text observation and external word meaning network (ConceptNet), then train a policy in the network with …
ANDRE: An Attention-based Neuro-symbolic Differentiable Rule Extractor for Inductive Logic Programming
Dept. of Mechanical and Aerospace Engineering, George Washington University, USA（机械与航空航天工程系，乔治华盛顿大学）；Dept. of Mechanical Engineering Sciences, University of Surrey, UK（机械工程科学系，萨里大学）
Recent work on neuro-symbolic inductive logic programming has led to promising approaches that can learn explanatory rules from noisy, real-world data. While some proposals approximate logical operators with differentiable operators from fuzzy or real-valued logic that are parameter-free thus diminishing their capacity to fit the data, other approaches are only loosely based on logic making it difficult to interpret the learned "rules". In this paper, we propose learning rules with the recently proposed logical neural networks (LNN). Compared to others, LNNs offer strong connection to classica…
Neural-Symbolic Logic Query Answering in Non-Euclidean Space
Institute for Clarity in Documentation（清晰文档研究所）；Inria Paris-Rocquencourt（巴黎-罗克琴堡研究所）；Rajiv Gandhi University（拉朱·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒尔研究实验室）
We introduce Natlog, a lightweight Logic Programming language, sharing Prolog's unification-driven execution model, but with a simplified syntax and semantics. Our proof-of-concept Natlog implementation is tightly embedded in the Python-based deep-learning ecosystem with focus on content-driven indexing of ground term datasets. As an overriding of our symbolic indexing algorithm, the same function can be delegated to a neural network, serving ground facts to Natlog's resolution engine. Our open-source implementation is available as a Python package at https://pypi.org/project/natlog/ .
University of Edinburgh（爱丁堡大学）；Alan Turing Institute（艾伦·图灵研究所）
The tension between deduction and induction is perhaps the most fundamental issue in areas such as philosophy, cognition and artificial intelligence (AI). The deduction camp concerns itself with questions about the expressiveness of formal languages for capturing knowledge about the world, together with proof systems for reasoning from such knowledge bases. The learning camp attempts to generalize from examples about partial descriptions about the world. In AI, historically, these camps have loosely divided the development of the field, but advances in cross-over areas such as statistical rela…
This paper presents Non-Axiomatic Term Logic (NATL) as a theoretical computational framework of humanlike symbolic reasoning in artificial intelligence. NATL unites a discrete syntactic system inspired from Aristotle's term logic and a continuous semantic system based on the modern idea of distributed representations, or embeddings. This paper positions the proposed approach in the phylogeny and the literature of logic, and explains the framework. As it is yet no more than a theory and it requires much further elaboration to implement it, no quantitative evaluation is presented. Instead, quali…
For many reasoning-heavy tasks involving raw inputs, it is challenging to design an appropriate end-to-end learning pipeline. Neuro-Symbolic Learning, divide the process into sub-symbolic perception and symbolic reasoning, trying to utilise data-driven machine learning and knowledge-driven reasoning simultaneously. However, they suffer from the exponential computational complexity within the interface between these two components, where the sub-symbolic learning model lacks direct supervision, and the symbolic model lacks accurate input facts. Hence, most of them assume the existence of a stro…
Departamento de Matemática（数学系）；Instituto Superior de Engenharia de Lisboa（里斯本高等工程学院）
This work describes a methodology to combine logic-based systems and connectionist systems. Our approach uses finite truth valued Łukasiewicz logic, where we take advantage of fact what in this type of logics every connective can be define by a neuron in an artificial network having by activation function the identity truncated to zero and one. This allowed the injection of first-order formulas in a network architecture, and also simplified symbolic rule extraction. Our method trains a neural network using Levenderg-Marquardt algorithm, where we restrict the knowledge dissemination in the netw…
We present Logical Optimal Actions (LOA), an action decision architecture of reinforcement learning applications with a neuro-symbolic framework which is a combination of neural network and symbolic knowledge acquisition approach for natural language interaction games. The demonstration for LOA experiments consists of a web-based interactive platform for text-based games and visualization for acquired knowledge for improving interpretability for trained rules. This demonstration also provides a comparison module with other neuro-symbolic approaches as well as non-symbolic state-of-the-art agen…
University of Bremen（不来梅大学）；City University London（伦敦城市大学）；University of Edinburgh（爱丁堡大学）；University of Luxembourg（卢森堡大学）；University of Amsterdam（阿姆斯特丹大学）
This article aims to achieve two goals: to show that probability is not the only way of dealing with uncertainty (and even more, that there are kinds of uncertainty which are for principled reasons not addressable with probabilistic means); and to provide evidence that logic-based methods can well support reasoning with uncertainty. For the latter claim, two paradigmatic examples are presented: Logic Programming with Kleene semantics for modelling reasoning from information in a discourse, to an interpretation of the state of affairs of the intended model, and a neural-symbolic implementation …
Modeling Endogenous Logic: Causal Neuro-Symbolic Reasoning Model for Explainable Multi-Behavior Recommendation
School of Computer Science and Engineering（计算机科学与工程学院）；Nanjing University of Science and Technology（南京理工大学）；School of Management（管理学院）；Hefei University of Technology（合肥工业大学）；School of Computer Science and Artificial Intelligence（计算机科学与人工智能学院）；Nanjing University of Finance and Economics（南京财经大学）；Department of Finance and Accounting（金融与会计系）；Saint Petersburg State University（圣彼得堡国立大学）；School of Computing（计算机学院）；Macquarie University（麦考瑞大学）
CentraleSupelec（巴黎中央理工-高等电力学院）；Université Paris Saclay（巴黎萨克雷大学）；Sorbonne Université（索邦大学）；Université d’Artois（阿尔图瓦大学）
Modal logics have proved useful for many reasoning tasks in symbolic artificial intelligence (AI), such as belief revision, spatial reasoning, among others. On the other hand, mathematical morphology (MM) is a theory for non-linear analysis of structures, that was widely developed and applied in image analysis. Its mathematical bases rely on algebra, complete lattices, topology. Strong links have been established between MM and mathematical logics, mostly modal logics. In this paper, we propose to further develop and generalize this link between mathematical morphology and modal logic from a t…
Constraints-Guided Diffusion Reasoner for Neuro-Symbolic Learning
Fudan University（复旦大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Academy of Artificial Intelligence for Science（上海人工智能科学研究院）；INFTECH；University of Waterloo（滑铁卢大学）
The Australian E-Health Research Center, CSIRO（澳大利亚电子健康研究中心，澳大利亚联邦科学与工业研究组织）
Symbolic has been long considered as a language of human intelligence while neural networks have advantages of robust computation and dealing with noisy data. The integration of neural-symbolic can offer better learning and reasoning while providing a means for interpretability through the representation of symbolic knowledge. Although previous works focus intensively on supervised feedforward neural networks, little has been done for the unsupervised counterparts. In this paper we show how to integrate symbolic knowledge into unsupervised neural networks. We exemplify our approach with knowle…
This work introduces a neuro-symbolic agent that combines deep reinforcement learning (DRL) with temporal logic (TL) to achieve systematic zero-shot, i.e., never-seen-before, generalisation of formally specified instructions. In particular, we present a neuro-symbolic framework where a symbolic module transforms TL specifications into a form that helps the training of a DRL agent targeting generalisation, while a neural module learns systematically to solve the given tasks. We study the emergence of systematic learning in different settings and find that the architecture of the convolutional l…
Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction
Technical University of Darmstadt（达姆施塔特工业大学）；National Research Center for Applied Cybersecurity（国家应用网络安全研究中心）；Eindhoven University of Technology（埃因霍温理工大学）；Hessian Center for AI (hessian.AI)（黑森人工智能中心）；German Research Center for AI (DFKI)（德国人工智能研究中心）
MIT（麻省理工学院）；DeepMind；NYU（纽约大学）；Facebook AI Research（Facebook人工智能研究院）
Human reasoning can often be understood as an interplay between two systems: the intuitive and associative ("System 1") and the deliberative and logical ("System 2"). Neural sequence models -- which have been increasingly successful at performing complex, structured tasks -- exhibit the advantages and failure modes of System 1: they are fast and learn patterns from data, but are often inconsistent and incoherent. In this work, we seek a lightweight, training-free means of improving existing System 1-like sequence models by adding System 2-inspired logical reasoning. We explore several variatio…
Faculty of Engineering, Free University of Bozen-Bolzano（博洛尼亚-博尔扎诺自由大学工程学院）；Department of Computer, Control and Management Engineering, Sapienza, Università di Roma（罗马大学计算机、控制与管理工程系）
Instituto Superior de Engenharia de Lisboa（里斯本高等工程学院）
The development of machine learning in particular and artificial intelligent in general has been strongly conditioned by the lack of an appropriate interface layer between deduction, abduction and induction. In this work we extend traditional algebraic specification methods in this direction. Here we assume that such interface for AI emerges from an adequate Neural-Symbolic integration. This integration is made for universe of discourse described on a Topos governed by a many-valued Łukasiewicz logic. Sentences are integrated in a symbolic knowledge base describing the problem domain, codified…
University of Edinburgh（爱丁堡大学）；Alan Turing Institute（艾伦·图灵研究所）
The unification of low-level perception and high-level reasoning is a long-standing problem in artificial intelligence, which has the potential to not only bring the areas of logic and learning closer together but also demonstrate how abstract concepts might emerge from sensory data. Precisely because deep learning methods dominate perception-based learning, including vision, speech, and linguistic grammar, there is fast-growing literature on how to integrate symbolic reasoning and deep learning. Broadly, efforts seem to fall into three camps: those focused on defining a logic whose formulas c…
Arizona State University（亚利桑那州立大学）；Universidad Nacional del Sur（国立南方大学）
While deep neural networks have led to major advances in image recognition, language translation, data mining, and game playing, there are well-known limits to the paradigm such as lack of explainability, difficulty of incorporating prior knowledge, and modularity. Neuro symbolic hybrid systems have recently emerged as a straightforward way to extend deep neural networks by incorporating ideas from symbolic reasoning such as computational logic. In this paper, we propose a list desirable criteria for neuro symbolic systems and examine how some of the existing approaches address these criteria.…
