Inductive logic programming (ILP) is a form of machine learning. The goal of ILP is to induce a hypothesis (a set of logical rules) that generalises training examples. As ILP turns 30, we provide a new introduction to the field. We introduce the necessary logical notation and the main learning settings; describe the building blocks of an ILP system; compare several systems on several dimensions; describe four systems (Aleph, TILDE, ASPAL, and Metagol); highlight key application areas; and, finally, summarise current limitations and directions for future research.
University of Oxford（牛津大学）；University of Cambridge（剑桥大学）
Multi-core machines are ubiquitous. However, most inductive logic programming (ILP) approaches use only a single core, which severely limits their scalability. To address this limitation, we introduce parallel techniques based on constraint-driven ILP where the goal is to accumulate constraints to restrict the hypothesis space. Our experiments on two domains (program synthesis and inductive general game playing) show that (i) parallelisation can substantially reduce learning times, and (ii) worker communication (i.e. sharing constraints) is important for good performance.
University of Oxford（牛津大学）；KU Leuven（鲁汶大学）；Imperial College London（伦敦帝国理工学院）
Inductive logic programming (ILP) is a form of logic-based machine learning. The goal is to induce a hypothesis (a logic program) that generalises given training examples. As ILP turns 30, we review the last decade of research. We focus on (i) new meta-level search methods, (ii) techniques for learning recursive programs, (iii) new approaches for predicate invention, and (iv) the use of different technologies. We conclude by discussing current limitations of ILP and directions for future research.
University of Oxford（牛津大学）；KU Leuven（鲁汶大学）；Imperial College London（帝国理工学院）
Common criticisms of state-of-the-art machine learning include poor generalisation, a lack of interpretability, and a need for large amounts of training data. We survey recent work in inductive logic programming (ILP), a form of machine learning that induces logic programs from data, which has shown promise at addressing these limitations. We focus on new methods for learning recursive programs that generalise from few examples, a shift from using hand-crafted background knowledge to \emph{learning} background knowledge, and the use of different technologies, notably answer set programming and…
Mila（米拉研究所）；McGill University（麦吉尔大学）；Université de Montréal（蒙特利尔大学）；Stanford University（斯坦福大学）；HEC Montréal（蒙特利尔高等商学院）
Formulating and answering logical queries is a standard communication interface for knowledge graphs (KGs). Alleviating the notorious incompleteness of real-world KGs, neural methods achieved impressive results in link prediction and complex query answering tasks by learning representations of entities, relations, and queries. Still, most existing query answering methods rely on transductive entity embeddings and cannot generalize to KGs containing new entities without retraining the entity embeddings. In this work, we study the inductive query answering task where inference is performed on a …
Building Rules on Top of Ontologies for the Semantic Web with Inductive Logic Programming
Università degli Studi di Bari（巴里大学）
在本体之上构建规则是语义Web逻辑层的最终目标。为此，目前正在讨论一种用于这一层的特别标记语言。它的目的是遵循传统的混合知识表示和推理系统，如$\mathcal{AL}$ -日志，集成了描述逻辑$\mathcal{ALC}$和无函数霍恩子句语言\textsc{Datalog}。在本文中，我们考虑的问题，这些规则的语义Web的自动化收购。我们提出了一个通用的规则归纳框架，采用归纳逻辑程序设计的方法装置，并依赖于$\mathcal{AL}$ -log的表达和演绎能力。无论归纳的范围（描述与预测）如何，该框架都是有效的。然而，为了说明的目的，我们还讨论了一个实例化的框架，其目的是描述和原来是有用的本体精化。关键词：归纳逻辑程序设计，混合知识表示和推理系统，本体，语义网。逻辑程序设计理论与实践（Theory and Practice of Logic Programming，TPLP）
The capability of making interpretable and self-explanatory decisions is essential for developing responsible machine learning systems. In this work, we study the learning to explain problem in the scope of inductive logic programming (ILP). We propose Neural Logic Inductive Learning (NLIL), an efficient differentiable ILP framework that learns first-order logic rules that can explain the patterns in the data. In experiments, compared with the state-of-the-art methods, we find NLIL can search for rules that are x10 times longer while remaining x3 times faster. We also show that NLIL can scale …
Inductive logic programming is a type of machine learning in which logic programs are learned from examples. This learning typically occurs relative to some background knowledge provided as a logic program. This dissertation introduces bottom preprocessing, a method for generating initial constraints on the programs an ILP system must consider. Bottom preprocessing applies ideas from inverse entailment to modern ILP systems. Inverse entailment is an influential early ILP approach introduced with Progol. This dissertation also presents $\bot$-Popper, an implementation of bottom preprocessing fo…
We introduce an inductive logic programming approach that combines classical divide-and-conquer search with modern constraint-driven search. Our anytime approach can learn optimal, recursive, and large programs and supports predicate invention. Our experiments on three domains (classification, inductive general game playing, and program synthesis) show that our approach can increase predictive accuracies and reduce learning times.
Recent work on neuro-symbolic inductive logic programming has led to promising approaches that can learn explanatory rules from noisy, real-world data. While some proposals approximate logical operators with differentiable operators from fuzzy or real-valued logic that are parameter-free thus diminishing their capacity to fit the data, other approaches are only loosely based on logic making it difficult to interpret the learned "rules". In this paper, we propose learning rules with the recently proposed logical neural networks (LNN). Compared to others, LNNs offer strong connection to classica…
Kyoto University（京都大学）；NTT Communication Science Laboratories（NTT通信科学实验室）
The differentiable implementation of logic yields a seamless combination of symbolic reasoning and deep neural networks. Recent research, which has developed a differentiable framework to learn logic programs from examples, can even acquire reasonable solutions from noisy datasets. However, this framework severely limits expressions for solutions, e.g., no function symbols are allowed, and the shapes of clauses are fixed. As a result, the framework cannot deal with structured examples. Therefore we propose a new framework to learn logic programs from noisy and structured examples, including th…
Universidad Autonoma de Madrid（马德里自治大学）；Laboratoire des Sciences du Numérique de Nantes（南特数字科学实验室）；National Institute of Informatics（国立信息学研究所）
Machine learning methods are growing in relevance for biometrics and personal information processing in domains such as forensics, e-health, recruitment, and e-learning. In these domains, white-box (human-readable) explanations of systems built on machine learning methods can become crucial. Inductive Logic Programming (ILP) is a subfield of symbolic AI aimed to automatically learn declarative theories about the process of data. Learning from Interpretation Transition (LFIT) is an ILP technique that can learn a propositional logic theory equivalent to a given black-box system (under certain co…
We present an update on the current architecture of the Zoea knowledge-based, Composable Inductive Programming system. The Zoea compiler is built using a modern variant of the black-board architecture. Zoea integrates a large number of knowledge sources that encode different aspects of programming language and software development expertise. We describe the use of synthetic test cases as a ubiquitous form of knowledge and hypothesis representation that sup-ports a variety of reasoning strategies. Some future plans are also outlined.
In recent research, human-understandable explanations of machine learning models have received a lot of attention. Often explanations are given in form of model simplifications or visualizations. However, as shown in cognitive science as well as in early AI research, concept understanding can also be improved by the alignment of a given instance for a concept with a similar counterexample. Contrasting a given instance with a structurally similar example which does not belong to the concept highlights what characteristics are necessary for concept membership. Such near misses have been proposed…
We propose a novel paradigm for solving Inductive Logic Programming (ILP) problems via deep recurrent neural networks. This proposed ILP solver is designed based on differentiable implementation of the deduction via forward chaining. In contrast to the majority of past methods, instead of searching through the space of possible first-order logic rules by using some restrictive rule templates, we directly learn the symbolic logical predicate rules by introducing a novel differentiable Neural Logic (dNL) network. The proposed dNL network is able to learn and represent Boolean functions efficient…
University of Klagenfurt（克拉根福大学）；Graz University of Technology（格拉茨工业大学）
Efficient omission of symmetric solution candidates is essential for combinatorial problem-solving. Most of the existing approaches are instance-specific and focus on the automatic computation of Symmetry Breaking Constraints (SBCs) for each given problem instance. However, the application of such approaches to large-scale instances or advanced problem encodings might be problematic since the computed SBCs are propositional and, therefore, can neither be meaningfully interpreted nor transferred to other instances. As a result, a time-consuming recomputation of SBCs must be done before every in…
Inductive Logic Programming in Databases: from Datalog to DL+log
Università degli Studi di Bari（巴里大学）
在本文中，我们解决了一个问题，已经引起了数据库社区的注意，随着语义Web的出现，即本体论（和它们所传达的语义）如何帮助解决典型的数据库问题的问题，通过更好地了解KR方面相关的数据库。特别是，我们调查这个问题，从ILP的角度考虑两个数据库的问题，（i）的定义的意见和（ii）的定义的约束，为数据库的模式表示也通过本体。两者都可以重新表述为ILP问题，并可以受益于KR框架DL+log的表达和演绎能力。我们通过实例说明了应用场景。关键词：归纳逻辑程序设计，关系数据库，本体，描述逻辑，混合知识表示和推理系统。逻辑程序设计理论与实践（Theory and Practice of Logic Programming，TPLP）
Explaining Neural Networks in Preference Learning: a Post-hoc Inductive Logic Programming Approach
Department of Information Engineering, Computer Science and Mathematics, University of L'Aquila, Italy（意大利拉奎拉大学信息工程、计算机科学与数学系）；Department of Life, Health and Environmental Sciences, University of L'Aquila, Italy（意大利拉奎拉大学生命、健康与环境科学系）；Department of Electrical Engineering and Information Technologies, University of Naples, Italy（意大利那不勒斯大学电气工程与信息技术系）；Ethos Group, Department of Human Sciences, University of Verona, Italy（意大利维罗纳大学人文科学系Ethos研究组）
From Circuit Evidence to Mechanistic Theory: An Inductive Logic Approach
Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）；Idiap Research Institute（Idiap研究机构）；CRUK National Biomarker Centre, University of Manchester（曼彻斯特大学癌症研究联盟国家生物标志物中心）
ANDRE: An Attention-based Neuro-symbolic Differentiable Rule Extractor for Inductive Logic Programming
Dept. of Mechanical and Aerospace Engineering, George Washington University, USA（机械与航空航天工程系，乔治华盛顿大学）；Dept. of Mechanical Engineering Sciences, University of Surrey, UK（机械工程科学系，萨里大学）
Technische Universität Wien（维也纳技术大学）；Marmara University（马尔马拉大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Sabanci University（萨班哲大学）
We describe the Inspire system which participated in the first competition on Inductive Logic Programming (ILP). Inspire is based on Answer Set Programming (ASP). The distinguishing feature of Inspire is an ASP encoding for hypothesis space generation: given a set of facts representing the mode bias, and a set of cost configuration parameters, each answer set of this encoding represents a single rule that is considered for finding a hypothesis that entails the given examples. Compared with state-of-the-art methods that use the length of the rule body as a metric for rule complexity, our approa…
Inductive Learning for Possibilistic Logic Programs Under Stable Models
Multi-dimensional data perception and intelligent recognition Chongqing Engineering Research Center, Chongqing University of Arts and Sciences, Chongqing, 402160, China（重庆艺术科技大学智能识别多维数据感知工程研究中心）；College of Computer Science and Technology, Guizhou University, Guiyang, 550025, China（贵州大学计算机科学与技术学院）；State Key Laboratory of Public Big Data（公共大数据国家重点实验室）；Key Laboratory of Advanced Medical Imaging（先进医学影像重点实验室）；Institute for Artificial Intelligence, Guizhou University, Guiyang, 550025, China（贵州大学人工智能研究所）；School of Information and Communication Technology, Griffith University, QLD 4111, Australia（格里菲斯大学信息与通信技术学院）
The goal of Inductive Logic Programming (ILP) is to learn a program that explains a set of examples. Until recently, most research on ILP targeted learning Prolog programs. The ILASP system instead learns Answer Set Programs (ASP). Learning such expressive programs widens the applicability of ILP considerably; for example, enabling preference learning, learning common-sense knowledge, including defaults and exceptions, and learning non-deterministic theories. Early versions of ILASP can be considered meta-level ILP approaches, which encode a learning task as a logic program and delegate the se…
Zoea Ltd（佐亚有限公司）；London South Bank University（伦敦南岸大学）
Automatic generation of software from some form of specification has been a long standing goal of computer science research. To date successful results have been reported for the production of relatively small programs. This paper presents Zoea which is a simple programming language that allows software to be generated from a specification format that closely resembles a set of automated functional tests. Zoea incorporates a number of advances that enable it to generate software that is large enough to have commercial value. Zoea also allows programs to be composed to form still larger program…
Humans constantly restructure knowledge to use it more efficiently. Our goal is to give a machine learning system similar abilities so that it can learn more efficiently. We introduce the \textit{knowledge refactoring} problem, where the goal is to restructure a learner's knowledge base to reduce its size and to minimise redundancy in it. We focus on inductive logic programming, where the knowledge base is a logic program. We introduce Knorf, a system which solves the refactoring problem using constraint optimisation. We evaluate our approach on two program induction domains: real-world string…
Sun Yat-sen University（中山大学）；Tencent AI Lab（腾讯人工智能实验室）；Wuhan University（武汉大学）
Knowledge graph embedding aims at modeling entities and relations with low-dimensional vectors. Most previous methods require that all entities should be seen during training, which is unpractical for real-world knowledge graphs with new entities emerging on a daily basis. Recent efforts on this issue suggest training a neighborhood aggregator in conjunction with the conventional entity and relation embeddings, which may help embed new entities inductively via their existing neighbors. However, their neighborhood aggregators neglect the unordered and unequal natures of an entity's neighbors. T…
Motivated by the needs of resource constrained dialog policy learning, we introduce dialog policy via differentiable inductive logic (DILOG). We explore the tasks of one-shot learning and zero-shot domain transfer with DILOG on SimDial and MultiWoZ. Using a single representative dialog from the restaurant domain, we train DILOG on the SimDial dataset and obtain 99+% in-domain test accuracy. We also show that the trained DILOG zero-shot transfers to all other domains with 99+% accuracy, proving the suitability of DILOG to slot-filling dialogs. We further extend our study to the MultiWoZ dataset…
Universitat Politècnica de València（瓦伦西亚理工大学）；University of Miyazaki（宫崎大学）
Given one or two examples, humans are good at understanding how to solve a problem independently of its domain, because they are able to detect what the problem is and to choose the appropriate background knowledge according to the context. For instance, presented with the string "8/17/2017" to be transformed to "17th of August of 2017", humans will process this in two steps: (1) they recognise that it is a date and (2) they map the date to the 17th of August of 2017. Inductive Programming (IP) aims at learning declarative (functional or logic) programs from examples. Two key advantages of IP …
ICLP is the premier international event for presenting research in logic programming. Contributions to ICLP 2022 were sought in all areas of logic programming, including but not limited to: Foundations: Semantics, Formalisms, Nonmonotonic reasoning, Knowledge representation. Languages issues: Concurrency, Objects, Coordination, Mobility, Higher order, Types, Modes, Assertions, Modules, Meta-programming, Logic-based domain-specific languages, Programming techniques. Programming support: Program analysis, Transformation, Validation, Verification, Debugging, Profiling, Testing, Execution visualiz…
