Symbolic Regression (SR) is a well-established framework for generating interpretable or white-box predictive models. Although SR has been successfully applied to create interpretable estimates of the average of the outcome, it is currently not well understood how it can be used to estimate the relationship between variables at other points in the distribution of the target variable. Such estimates of e.g. the median or an extreme value provide a fuller picture of how predictive variables affect the outcome and are necessary in high-stakes, safety-critical application domains. This study introduces Symbolic Quantile Regression (SQR), an approach to predict conditional quantiles with SR. In an extensive evaluation, we find that SQR outperforms transparent models and performs comparably to a strong black-box baseline without compromising transparency. We also show how SQR can be used to explain differences in the target distribution by comparing models that predict extreme and central outcomes in an airline fuel usage case study. We conclude that SQR is suitable for predicting conditional quantiles and understanding interesting feature influences at varying quantiles.
Based on the novel concept of multivariate center-outward quantiles introduced recently in Chernozhukov et al. (2017) and Hallin et al. (2021), we are considering the problem of nonparametric multiple-output quantile regression. Our approach defines nested conditional center-outward quantile regression contours and regions with given conditional probability content irrespective of the underlying distribution; their graphs constitute nested center-outward quantile regression tubes. Empirical counterparts of these concepts are constructed, yielding interpretable empirical regions and contours wh…
Motivated by the need for effectively summarising, modelling, and forecasting the distributional characteristics of intra-daily returns, as well as the recent work on forecasting histogram-valued time-series in the area of symbolic data analysis, we develop a time-series model for forecasting quantile-function-valued (QF-valued) daily summaries for intra-daily returns. We call this model the dynamic quantile function (DQF) model. Instead of a histogram, we propose to use a $g$-and-$h$ quantile function to summarise the distribution of intra-daily returns. We work with a Bayesian formulation of…
Statistical techniques used in air pollution modelling usually lack the possibility to understand which predictors affect air pollution in which functional form; and are not able to regress on exceedances over certain thresholds imposed by authorities directly. The latter naturally induce conditional quantiles and reflect the seriousness of particular events. In the present paper we focus on this important aspect by developing quantile regression models further. We propose a general Bayesian effect selection approach for additive quantile regression within a highly interpretable framework. We …
Regression modelling of spatiotemporal extreme U.S. wildfires via partially-interpretable neural networks
School of Mathematics, University of Edinburgh（爱丁堡大学数学学院）；Statistics Program, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学计算机、电气与数学科学与工程（CEMSE）分部统计项目）
Deep learning has enjoyed tremendous success in a variety of applications but its application to quantile regressions remains scarce. A major advantage of the deep learning approach is its flexibility to model complex data in a more parsimonious way than nonparametric smoothing methods. However, while deep learning brought breakthroughs in prediction, it often lacks interpretability due to the black-box nature of multilayer structure with millions of parameters, hence it is not well suited for statistical inference. In this paper, we leverage the advantages of deep learning to apply it to quan…
Binary regression models are ubiquitous in virtually every scientific field. Frequently, traditional generalized linear models fail to capture the variability in the probability surface that gives rise to the binary observations and novel methodology is required. This has generated a substantial literature comprised of binary regression models motivated by various applications. We describe a novel organization of generalizations to traditional binary regression methods based on the familiar three-part structure of generalized linear models (random component, systematic component, link function…
Towards symbolic regression for interpretable clinical decision scores
Federal University of ABC（ABC联邦大学）；University of Pennsylvania（宾夕法尼亚大学）；Institute for Biomedical Informatics, University of Pennsylvania（宾夕法尼亚大学生物医学信息学研究所）；Boston Children's Hospital, Harvard Medical School（哈佛医学院波士顿儿童医院）
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Singapore Eye Research Institute（新加坡眼科研究所）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）；Institute for Molecular and Clinical Ophthalmology（分子与临床眼科研究所）；National University of Singapore（新加坡国立大学）
In this paper, we propose an explainable and interpretable diabetic retinopathy (ExplainDR) classification model based on neural-symbolic learning. To gain explainability, a highlevel symbolic representation should be considered in decision making. Specifically, we introduce a human-readable symbolic representation, which follows a taxonomy style of diabetic retinopathy characteristics related to eye health conditions to achieve explainability. We then include humanreadable features obtained from the symbolic representation in the disease prediction. Experimental results on a diabetic retinopa…
Rensselaer Polytechnic Institute（伦斯勒理工学院）；IBM T.J. Watson Research Center（IBM托马斯·J·沃森研究中心）；IBM Research（IBM研究院）
Most existing Time series classification (TSC) models lack interpretability and are difficult to inspect. Interpretable machine learning models can aid in discovering patterns in data as well as give easy-to-understand insights to domain specialists. In this study, we present Neuro-Symbolic Time Series Classification (NSTSC), a neuro-symbolic model that leverages signal temporal logic (STL) and neural network (NN) to accomplish TSC tasks using multi-view data representation and expresses the model as a human-readable, interpretable formula. In NSTSC, each neuron is linked to a symbolic express…
National Aeronautics and Space Administration（美国国家航空航天局）；University of Utah（犹他大学）
Interpretability and uncertainty quantification in machine learning can provide justification for decisions, promote scientific discovery and lead to a better understanding of model behavior. Symbolic regression provides inherently interpretable machine learning, but relatively little work has focused on the use of symbolic regression on noisy data and the accompanying necessity to quantify uncertainty. A new Bayesian framework for genetic-programming-based symbolic regression (GPSR) is introduced that uses model evidence (i.e., marginal likelihood) to formulate replacement probability during …
Deep neural networks based on layer-stacking architectures have historically suffered from poor inherent interpretability. Meanwhile, symbolic probabilistic models function with clear interpretability, but how to combine them with neural networks to enhance their performance remains to be explored. In this paper, we try to marry these two systems for text classification via a structured language model. We propose a Symbolic-Neural model that can learn to explicitly predict class labels of text spans from a constituency tree without requiring any access to span-level gold labels. As the structu…
Massive language models are the core of modern NLP modeling and have been shown to encode impressive amounts of commonsense and factual information. However, that knowledge exists only within the latent parameters of the model, inaccessible to inspection and interpretation, and even worse, factual information memorized from the training corpora is likely to become stale as the world changes. Knowledge stored as parameters will also inevitably exhibit all of the biases inherent in the source materials. To address these problems, we develop a neural language model that includes an explicit inter…
Facebook AI Research（脸书人工智能研究院）；Georgia Tech（佐治亚理工学院）
We propose a new class of probabilistic neural-symbolic models, that have symbolic functional programs as a latent, stochastic variable. Instantiated in the context of visual question answering, our probabilistic formulation offers two key conceptual advantages over prior neural-symbolic models for VQA. Firstly, the programs generated by our model are more understandable while requiring lesser number of teaching examples. Secondly, we show that one can pose counterfactual scenarios to the model, to probe its beliefs on the programs that could lead to a specified answer given an image. Our resu…
IBM Research Europe（IBM欧洲研究院）；Imperial College London（伦敦帝国理工学院）；ILASP Limited（ILASP有限公司）；ICREA-Universitat Pompeu Fabra（ICREA-庞培法布拉大学）
Logic-based machine learning aims to learn general, interpretable knowledge in a data-efficient manner. However, labelled data must be specified in a structured logical form. To address this limitation, we propose a neural-symbolic learning framework, called Feed-Forward Neural-Symbolic Learner (FFNSL), that integrates a logic-based machine learning system capable of learning from noisy examples, with neural networks, in order to learn interpretable knowledge from labelled unstructured data. We demonstrate the generality of FFNSL on four neural-symbolic classification problems, where different…
Turning Time Series into Algebraic Equations: Symbolic Machine Learning for Interpretable Modeling of Chaotic Time Series
SAFIR, Sorbonne University Abu Dhabi, UAE（SAFIR，索邦大学阿布扎赫德分校）；Sorbonne Center for Artificial Intelligence, Sorbonne University, Paris, France（索邦人工智能中心，索邦大学巴黎分校）
Symbolic--KAN: Kolmogorov-Arnold Networks with Discrete Symbolic Structure for Interpretable Learning
Energy \& Intelligence Lab, Department of Chemical Engineering, University of Utah, Salt Lake City, Utah, 84112, USA；Department of Mechanical Engineering, University of Utah, Salt Lake City, Utah, 84112, USA；Imaging Institute, University of Utah, Salt Lake City, Utah, 84112, USA；Department of Biomedical Engineering, University of Utah, Salt Lake City, Utah, 84112, USA；Department of Mechanical Engineering, School of Engineering, Urmia University of Technology, Urmia, Iran
Recently deep reinforcement learning has achieved tremendous success in wide ranges of applications. However, it notoriously lacks data-efficiency and interpretability. Data-efficiency is important as interacting with the environment is expensive. Further, interpretability can increase the transparency of the black-box-style deep RL models and hence gain trust from the users. In this work, we propose a new hierarchical framework via symbolic RL, leveraging a symbolic transition model to improve the data-efficiency and introduce the interpretability for learned policy. This framework consists o…
Discovering an interpretable mathematical expression for a full wind-turbine wake with artificial intelligence enhanced symbolic regression
School of Ocean and Civil Engineering, Shanghai Jiao Tong University（上海交通大学海洋与土木工程学院）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工大学宁波数字孪生研究院）
Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction
Technical University of Darmstadt（达姆施塔特工业大学）；National Research Center for Applied Cybersecurity（国家应用网络安全研究中心）；Eindhoven University of Technology（埃因霍温理工大学）；Hessian Center for AI (hessian.AI)（黑森人工智能中心）；German Research Center for AI (DFKI)（德国人工智能研究中心）
UIUC（伊利诺伊大学厄巴纳-香槟分校）；U of Oregon（俄勒冈大学）；MIT-IBM Watson AI Lab & IBM Research（MIT-IBM沃森人工智能实验室与IBM研究院）；Picsart AI Research (PAIR)（Picsart AI研究（PAIR））
We study the problem of concept induction in visual reasoning, i.e., identifying concepts and their hierarchical relationships from question-answer pairs associated with images; and achieve an interpretable model via working on the induced symbolic concept space. To this end, we first design a new framework named object-centric compositional attention model (OCCAM) to perform the visual reasoning task with object-level visual features. Then, we come up with a method to induce concepts of objects and relations using clues from the attention patterns between objects' visual features and question…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
We consider the problem of combining machine learning models to perform higher-level cognitive tasks with clear specifications. We propose the novel problem of Visual Discrimination Puzzles (VDP) that requires finding interpretable discriminators that classify images according to a logical specification. Humans can solve these puzzles with ease and they give robust, verifiable, and interpretable discriminators as answers. We propose a compositional neurosymbolic framework that combines a neural network to detect objects and relationships with a symbolic learner that finds interpretable discrim…
Interpretable Robotic Friction Learning via Symbolic Regression
Ludwig-Maximilians-Universität München（慕尼黑路易斯-马克西米利安大学）；German Aerospace Center（德国航空航天中心）；University of Tromsø（特罗姆瑟大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Structured Kolmogorov-Arnold Neural ODEs for Interpretable Learning and Symbolic Discovery of Nonlinear Dynamics
Department of Industrial Systems Engineering and Management, National University of Singapore（工业系统工程与管理系，新加坡国立大学）；Department of Civil, Environmental and Geomatic Engineering, ETH Zürich（土木、环境与测绘工程系，苏黎世联邦理工学院）；Future Resilient Systems, Singapore-ETH Centre（未来韧性系统，新加坡-苏黎世中心）
Learning Interpretable Network Dynamics via Universal Neural Symbolic Regression
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University（吉林大学符号计算与知识工程教育部重点实验室）
University of Cambridge（剑桥大学）；Université Côte d’Azur（蔚蓝海岸大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；University of Siena（锡耶纳大学）；INRA（法国国家农业食品与环境研究院）；Université Paris-Saclay（巴黎萨克雷大学）；KU Leuven（鲁汶大学）
Closing the Loop: PID Feedback Control for Interpretable Activation Steering in Symbolic Music Generation
Athens University of Economics and Business（雅典经济与商业大学）；Orfium Research（Orfium 研究）；Hellenic Mediterranean University（希腊地中海大学）；Archimedes / Athena Research Center（阿基米德/雅典娜研究中心）
Insight Centre for Data Analytics（洞察数据分析中心）；University College Dublin（都柏林大学学院）
The time series classification literature has expanded rapidly over the last decade, with many new classification approaches published each year. The research focus has mostly been on improving the accuracy and efficiency of classifiers, while their interpretability has been somewhat neglected. Classifier interpretability has become a critical constraint for many application domains and the introduction of the 'right to explanation' GDPR EU legislation in May 2018 is likely to further emphasize the importance of explainable learning algorithms. In this work we analyse the state-of-the-art for …
