Humans can learn new concepts from a small number of examples by drawing on their inductive biases. These inductive biases have previously been captured by using Bayesian models defined over symbolic hypothesis spaces. Is it possible to create a neural network that displays the same inductive biases? We show that inductive biases that enable rapid concept learning can be instantiated in artificial neural networks by distilling a prior distribution from a symbolic Bayesian model via meta-learning, an approach for extracting the common structure from a set of tasks. By generating the set of tasks used in meta-learning from the prior distribution of a Bayesian model, we are able to transfer that prior into a neural network. We use this approach to create a neural network with an inductive bias towards concepts expressed as short logical formulas. Analyzing results from previous behavioral experiments in which people learned logical concepts from a few examples, we find that our meta-trained models are highly aligned with human performance.
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
Machine learning applications to symbolic mathematics are becoming increasingly popular, yet there lacks a centralized source of real-world symbolic expressions to be used as training data. In contrast, the field of natural language processing leverages resources like Wikipedia that provide enormous amounts of real-world textual data. Adopting the philosophy of "mathematics as language," we bridge this gap by introducing a pipeline for distilling mathematical expressions embedded in Wikipedia into symbolic encodings to be used in downstream machine learning tasks. We demonstrate that a $\texti…
Closed-Form Interpretation of Neural Network Latent Spaces with Symbolic Gradients
University of Waterloo（多伦多大学）；Perimeter Institute for Theoretical Physics（理论物理研究所）；Homes Plus Magazine Inc.（Homes Plus 杂志公司）；University of Toronto（多伦多大学）
Siemens AG（西门子股份公司）；Technical University of Munich（慕尼黑工业大学）
Similarly to other connectionist models, Graph Neural Networks (GNNs) lack transparency in their decision-making. A number of sub-symbolic approaches have been developed to provide insights into the GNN decision making process. These are first important steps on the way to explainability, but the generated explanations are often hard to understand for users that are not AI experts. To overcome this problem, we introduce a conceptual approach combining sub-symbolic and symbolic methods for human-centric explanations, that incorporate domain knowledge and causality. We furthermore introduce the …
SymbOmni: Evolving Agentic Omni Models via Symbolic Concept Learning
South China University of Technology（华南理工大学）；Westlake University（西湖大学）；Johns Hopkins University（约翰·霍普金斯大学）；The Chinese University of Hong Kong（香港中文大学）；Shenzhen Loop Area Institute（深圳河套学院）
Many complex problems are naturally understood in terms of symbolic concepts. For example, our concept of "cat" is related to our concepts of "ears" and "whiskers" in a non-arbitrary way. Fodor (1998) proposes one theory of concepts, which emphasizes symbolic representations related via constituency structures. Whether neural networks are consistent with such a theory is open for debate. We propose unit tests for evaluating whether a system's behavior is consistent with several key aspects of Fodor's criteria. Using a simple visual concept learning task, we evaluate several modern neural archi…
University of California, Los Angeles（加州大学洛杉矶分校）；Peking University（北京大学）
This paper develops a novel methodology for using symbolic knowledge in deep learning. From first principles, we derive a semantic loss function that bridges between neural output vectors and logical constraints. This loss function captures how close the neural network is to satisfying the constraints on its output. An experimental evaluation shows that it effectively guides the learner to achieve (near-)state-of-the-art results on semi-supervised multi-class classification. Moreover, it significantly increases the ability of the neural network to predict structured objects, such as rankings a…
Motivated by recent findings from cognitive neural science, we advocate the use of a dual-level model for concept representations: the embodied level consists of concept-oriented feature representations, and the symbolic level consists of concept graphs. Embodied concept representations are modality specific and exist in the form of feature vectors in a feature space. Symbolic concept representations, on the other hand, are amodal and language specific, and exist in the form of word / knowledge-graph embeddings in a concept / knowledge space. The human conceptual system comprises both embodied…
We propose neural-symbolic integration for abstract concept explanation and interactive learning. Neural-symbolic integration and explanation allow users and domain-experts to learn about the data-driven decision making process of large neural models. The models are queried using a symbolic logic language. Interaction with the user then confirms or rejects a revision of the neural model using logic-based constraints that can be distilled into the model architecture. The approach is illustrated using the Logic Tensor Network framework alongside Concept Activation Vectors and applied to a Convol…
GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learning
Technical University of Darmstadt（达姆施塔特技术大学）；Google Intrinsic（谷歌内在）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；Centre for Cognitive Science, TU Darmstadt（达姆施塔特技术大学认知科学中心）
University of Trento（特伦托大学）；University of Pisa（比萨大学）
Neuro-symbolic predictors learn a mapping from sub-symbolic inputs to higher-level concepts and then carry out (probabilistic) logical inference on this intermediate representation. This setup offers clear advantages in terms of consistency to symbolic prior knowledge, and is often believed to provide interpretability benefits in that - by virtue of complying with the knowledge - the learned concepts can be better understood by human stakeholders. However, it was recently shown that this setup is affected by reasoning shortcuts whereby predictions attain high accuracy by leveraging concepts wi…
TU Darmstadt（达姆施塔特工业大学）；Hessian Center for AI (hessian.AI)（黑森人工智能中心）
Most explanation methods in deep learning map importance estimates for a model's prediction back to the original input space. These "visual" explanations are often insufficient, as the model's actual concept remains elusive. Moreover, without insights into the model's semantic concept, it is difficult -- if not impossible -- to intervene on the model's behavior via its explanations, called Explanatory Interactive Learning. Consequently, we propose to intervene on a Neuro-Symbolic scene representation, which allows one to revise the model on the semantic level, e.g. "never focus on the color to…
BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts
DISI, University of Trento（特伦托大学DISI学院）；DI, University of Pisa（比萨大学DI学院）；University of Edinburgh（爱丁堡大学）；CIMeC&DISI, University of Trento（特伦托大学CIMeC&DISI学院）
The human brain has many remarkable information processing characteristics that deeply puzzle scientists and engineers. Among the most important and the most intriguing of these characteristics are the brain's broad universality as a learning system and its mysterious ability to dynamically change (reconfigure) its behavior depending on a combinatorial number of different contexts. This paper discusses a class of hypothetically brain-like dynamically reconfigurable associative learning systems that shed light on the possible nature of these brain's properties. The systems are arranged on the g…
Concise and Logically Consistent Conformal Sets for Neuro-Symbolic Concept-Based Models
Department of Information Engineering and Computer Science, University of Trento, Italy（特伦托大学信息工程与计算机科学系）；CIMeC, University of Trento, Rovereto, Italy（特伦托大学罗韦雷托CIMeC）
Center for Neural Science（神经科学中心）；New York University（纽约大学）；Department of Psychology and Center for Data Science（心理学系与数据科学中心）
Human conceptual knowledge supports the ability to generate novel yet highly structured concepts, and the form of this conceptual knowledge is of great interest to cognitive scientists. One tradition has emphasized structured knowledge, viewing concepts as embedded in intuitive theories or organized in complex symbolic knowledge structures. A second tradition has emphasized statistical knowledge, viewing conceptual knowledge as an emerging from the rich correlational structure captured by training neural networks and other statistical models. In this paper, we explore a synthesis of these two …
A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts
DISI, University of Trento（特伦托大学DISI学院）；DI, University of Pisa（比萨大学DI学院）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）；University of Padova（帕多瓦大学）；University of Edinburgh（爱丁堡大学）；CIMeC, University of Trento（特伦托大学CIMeC中心）
Stanford University（斯坦福大学）；California Institute of Technology（加州理工学院）
Humans have the remarkable ability to recognize and acquire novel visual concepts in a zero-shot manner. Given a high-level, symbolic description of a novel concept in terms of previously learned visual concepts and their relations, humans can recognize novel concepts without seeing any examples. Moreover, they can acquire new concepts by parsing and communicating symbolic structures using learned visual concepts and relations. Endowing these capabilities in machines is pivotal in improving their generalization capability at inference time. In this work, we introduce Zero-shot Concept Recognit…
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；IIIS, Tsinghua University（清华大学交叉信息研究院）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；Deepmind（深度思考）；MIT BCS（麻省理工学院脑与认知科学系）；CBMM（脑、心智与机器中心）
We propose the Neuro-Symbolic Concept Learner (NS-CL), a model that learns visual concepts, words, and semantic parsing of sentences without explicit supervision on any of them; instead, our model learns by simply looking at images and reading paired questions and answers. Our model builds an object-based scene representation and translates sentences into executable, symbolic programs. To bridge the learning of two modules, we use a neuro-symbolic reasoning module that executes these programs on the latent scene representation. Analogical to human concept learning, the perception module learns…
Symbol Grounding in Neuro-Symbolic AI: A Gentle Introduction to Reasoning Shortcuts
University of Trento（特伦托大学）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；Sapienza University of Rome（罗马大学）；University of Edinburgh（爱丁堡大学）；Huawei Labs（华为实验室）
Despite rapid progress, current deep learning methods face a number of critical challenges. These include high energy consumption, catastrophic forgetting, dependance on global losses, and an inability to reason symbolically. By combining concepts from information bottleneck theory and vector-symbolic architectures, we propose and implement a novel information processing architecture, the 'Bridge network.' We show this architecture provides unique advantages which can address the problem of global losses and catastrophic forgetting. Furthermore, we argue that it provides a further basis for in…
UIUC（伊利诺伊大学厄巴纳-香槟分校）；U of Oregon（俄勒冈大学）；MIT-IBM Watson AI Lab & IBM Research（MIT-IBM沃森人工智能实验室与IBM研究院）；Picsart AI Research (PAIR)（Picsart AI研究（PAIR））
We study the problem of concept induction in visual reasoning, i.e., identifying concepts and their hierarchical relationships from question-answer pairs associated with images; and achieve an interpretable model via working on the induced symbolic concept space. To this end, we first design a new framework named object-centric compositional attention model (OCCAM) to perform the visual reasoning task with object-level visual features. Then, we come up with a method to induce concepts of objects and relations using clues from the attention patterns between objects' visual features and question…
People can learn rich, general-purpose conceptual representations from only raw perceptual inputs. Current machine learning approaches fall well short of these human standards, although different modeling traditions often have complementary strengths. Symbolic models can capture the compositional and causal knowledge that enables flexible generalization, but they struggle to learn from raw inputs, relying on strong abstractions and simplifying assumptions. Neural network models can learn directly from raw data, but they struggle to capture compositional and causal structure and typically must …
Dual embodied-symbolic concept representations are the foundation for deep learning and symbolic AI integration. We discuss the use of dual embodied-symbolic concept representations for molecular graph representation learning, specifically with exemplar-based contrastive self-supervised learning (SSL). The embodied representations are learned from molecular graphs, and the symbolic representations are learned from the corresponding Chemical knowledge graph (KG). We use the Chemical KG to enhance molecular graphs with symbolic (semantic) knowledge and generate their augmented molecular graphs. …
Humans have the ability to rapidly understand rich combinatorial concepts from limited data. Here we investigate this ability in the context of auditory signals, which have been evolved in a cultural transmission experiment to study the emergence of combinatorial structure in language. We propose a neuro-symbolic generative model which combines the strengths of previous approaches to concept learning. Our model performs fast inference drawing on neural network methods, while still retaining the interpretability and generalization from limited data seen in structured generative approaches. This…
Pre-trained transformers have recently clinched top spots in the gamut of natural language tasks and pioneered solutions to software engineering tasks. Even information retrieval has not been immune to the charm of the transformer, though their large size and cost is generally a barrier to deployment. While there has been much work in streamlining, caching, and modifying transformer architectures for production, here we explore a new direction: distilling a large pre-trained translation model into a lightweight bi-encoder which can be efficiently cached and queried. We argue from a probabilist…
Learning Concept-Based Causal Transition and Symbolic Reasoning for Visual Planning
Beijing Institute for General Artificial Intelligence (BIGAI)（北京通用人工智能研究院（BIGAI））；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；Yuanpei College, Peking University（北京大学元培学院）；University of California, Los Angeles (UCLA)（加利福尼亚大学洛杉矶分校）
University of Cambridge（剑桥大学）；Université Côte d’Azur（蔚蓝海岸大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；University of Siena（锡耶纳大学）；INRA（法国国家农业食品与环境研究院）；Université Paris-Saclay（巴黎萨克雷大学）；KU Leuven（鲁汶大学）
APPCAIR, BITS Pilani K.K. Birla Goa Campus（BITS Pilani K.K. Birla Goa校区APPCAIR）；TCS Research（塔塔咨询服务研究部）；IIT Delhi（印度理工学院德里分校）
We model short-duration (e.g. day) trading in financial markets as a sequential decision-making problem under uncertainty, with the added complication of continual concept-drift. We, therefore, employ meta reinforcement learning via the RL2 algorithm. It is also known that human traders often rely on frequently occurring symbolic patterns in price series. We employ logical program induction to discover symbolic patterns that occur frequently as well as recently, and explore whether using such features improves the performance of our meta reinforcement learning algorithm. We report experiments …
We introduce a neural network architecture and a learning algorithm to produce factorized symbolic representations. We propose to learn these concepts by observing consecutive frames, letting all the components of the hidden representation except a small discrete set (gating units) be predicted from the previous frame, and let the factors of variation in the next frame be represented entirely by these discrete gated units (corresponding to symbolic representations). We demonstrate the efficacy of our approach on datasets of faces undergoing 3D transformations and Atari 2600 games.
University of Maryland, Baltimore County（马里兰大学巴尔的摩郡分校）；Laboratory for Physical Sciences（物理科学实验室）；Booz Allen Hamilton（博思艾伦汉密尔顿公司）
Holographic Reduced Representations (HRR) are a method for performing symbolic AI on top of real-valued vectors by associating each vector with an abstract concept, and providing mathematical operations to manipulate vectors as if they were classic symbolic objects. This method has seen little use outside of older symbolic AI work and cognitive science. Our goal is to revisit this approach to understand if it is viable for enabling a hybrid neural-symbolic approach to learning as a differentiable component of a deep learning architecture. HRRs today are not effective in a differentiable soluti…
IIT Delhi（印度理工学院德里分校）；Google Research India（谷歌印度研究院）
Neural models and symbolic algorithms have recently been combined for tasks requiring both perception and reasoning. Neural models ground perceptual input into a conceptual vocabulary, on which a classical reasoning algorithm is applied to generate output. A key limitation is that such neural-to-symbolic models can only be trained end-to-end for tasks where the output space is symbolic. In this paper, we study neural-symbolic-neural models for reasoning tasks that require a conversion from an image input (e.g., a partially filled sudoku) to an image output (e.g., the image of the completed sud…
Deep reinforcement learning (RL) methods often require many trials before convergence, and no direct interpretability of trained policies is provided. In order to achieve fast convergence and interpretability for the policy in RL, we propose a novel RL method for text-based games with a recent neuro-symbolic framework called Logical Neural Network, which can learn symbolic and interpretable rules in their differentiable network. The method is first to extract first-order logical facts from text observation and external word meaning network (ConceptNet), then train a policy in the network with …
City University of Hong Kong（香港城市大学）；Tsinghua University（清华大学）；Microsoft Research Asia（微软亚洲研究院）；Chinese Academy of Sciences（中国科学院）
One-shot weight sharing methods have recently drawn great attention in neural architecture search due to high efficiency and competitive performance. However, weight sharing across models has an inherent deficiency, i.e., insufficient training of subnetworks in hypernetworks. To alleviate this problem, we present a simple yet effective architecture distillation method. The central idea is that subnetworks can learn collaboratively and teach each other throughout the training process, aiming to boost the convergence of individual models. We introduce the concept of prioritized path, which refer…
Facebook AI Research（脸书人工智能研究院）；Georgia Tech（佐治亚理工学院）
We propose a new class of probabilistic neural-symbolic models, that have symbolic functional programs as a latent, stochastic variable. Instantiated in the context of visual question answering, our probabilistic formulation offers two key conceptual advantages over prior neural-symbolic models for VQA. Firstly, the programs generated by our model are more understandable while requiring lesser number of teaching examples. Secondly, we show that one can pose counterfactual scenarios to the model, to probe its beliefs on the programs that could lead to a specified answer given an image. Our resu…
Compositionality in Time Series: A Proof of Concept using Symbolic Dynamics and Compositional Data Augmentation
Department of Computational Linguistics（计算语言学系）；Heidelberg University（海德堡大学）；Department of Computational Linguistics and Interdisciplinary Center for Scientific Computing (IWR)（计算语言学系和科学计算跨学科中心）
