Recently proposed models which learn to write computer programs from data use either input/output examples or rich execution traces. Instead, we argue that a novel alternative is to use a glass-box loss function, given as a program itself that can be directly inspected. Glass-box optimization covers a wide range of problems, from computing the greatest common divisor of two integers, to learning-to-learn problems.
In this paper, we present an intelligent search system which learns, given the partial program and the glass-box problem, the probabilities over the space of programs. We empirically demonstrate that our informed search procedure leads to significant improvements compared to brute-force program search, both in terms of accuracy and time. For our experiments we use rich context free grammars inspired by number theory, text processing, and algebra. Our results show that (i) performing 4 rounds of our framework typically solves about 70% of the target problems, (ii) our framework can improve itself even in domain agnostic scenarios, and (iii) it can solve problems that would be otherwise too slow to solve with brute-force search.
LLMs Understand Glass-Box Models, Discover Surprises, and Suggest Repairs
Massachusetts Institute of Technology（麻省理工学院）；Broad Institute of MIT and Harvard（麻省理工学院和哈佛大学布罗德研究所）；University of Tübingen（图宾根大学）；Microsoft Research（微软研究院）；NYU Langone Health（纽约大学朗格尼医学中心）
The increasing usage of machine learning models raises the question of the reliability of these models. The current practice of testing with limited data is often insufficient. In this paper, we provide a framework for automated test data synthesis to test black-box ML/DL models. We address an important challenge of generating realistic user-controllable data with model agnostic coverage criteria to test a varied set of properties, essentially to increase trust in machine learning models. We experimentally demonstrate the effectiveness of our technique.
Although researchers accumulated knowledge about seismogenesis and decades-long earthquake data, predicting imminent individual earthquakes at a specific time and location remains a long-standing enigma. This study hypothesizes that the observed data conceal the hidden rules which may be unraveled by a novel glass-box (as opposed to black-box) physics rule learner (GPRL) framework. Without any predefined earthquake-related mechanisms or statistical laws, GPRL's two essentials, convolved information index and transparent link function, seek generic expressions of rules directly from data. GPRL'…
By their nature, the composition of black box models is opaque. This makes the ability to generate explanations for the response to stimuli challenging. The importance of explaining black box models has become increasingly important given the prevalence of AI and ML systems and the need to build legal and regulatory frameworks around them. Such explanations can also increase trust in these uncertain systems. In our paper we present RICE, a method for generating explanations of the behaviour of black box models by (1) probing a model to extract model output examples using sensitivity analysis; …
CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全研究中心）；DeepMind（深度思维）
In this work, we investigate the problem of revealing the functionality of a black-box agent. Notably, we are interested in the interpretable and formal description of the behavior of such an agent. Ideally, this description would take the form of a program written in a high-level language. This task is also known as reverse engineering and plays a pivotal role in software engineering, computer security, but also most recently in interpretability. In contrast to prior work, we do not rely on privileged information on the black box, but rather investigate the problem under a weaker assumption o…
The recent surge in earthquake engineering is the use of machine learning methods to develop predictive models for structural behavior. Complex black-box models are typically used for decision making to achieve high accuracy; however, as important as high accuracy, it is essential for engineers to understand how the model makes the decision and verify that the model is physically meaningful. With this motivation, this study proposes a glass-box (interpretable) classification model to predict the seismic failure mode of conventional reinforced concrete shear (structural) walls. Reported experim…
University of Edinburgh（爱丁堡大学）；UC Berkeley（加州大学伯克利分校）；University of Iowa（爱荷华大学）
In classic program synthesis algorithms, such as counterexample-guided inductive synthesis (CEGIS), the algorithms alternate between a synthesis phase and an oracle (verification) phase. Many synthesis algorithms use a white-box oracle based on satisfiability modulo theory (SMT) solvers to provide counterexamples. But what if a white-box oracle is either not available or not easy to work with? We present a framework for solving a general class of oracle-guided synthesis problems which we term synthesis modulo oracles. In this setting, oracles may be black boxes with a query-response interface …
Indian Institute of Technology Hyderabad（印度理工学院海得拉巴分校）；Indian Institute of Information Technology Vadodara（印度信息技术学院瓦多达拉分校）
With the substantial performance of neural networks in sensitive fields increases the need for interpretable deep learning models. Major challenge is to uncover the multiscale and distributed representation hidden inside the basket mappings of the deep neural networks. Researchers have been trying to comprehend it through visual analysis of features, mathematical structures, or other data-driven approaches. Here, we work on implementation invariances of CNN-based representations and present an analytical binary prototype that provides useful insights for large scale real-life applications. We …
Opening the AI black box: program synthesis via mechanistic interpretability
Massachusetts Institute of Technology（麻省理工学院）；Institute for Artificial Intelligence and Fundamental Interactions（人工智能与基本相互作用研究所）；Harvard University（哈佛大学）
Deep reinforcement learning has led to several recent breakthroughs, though the learned policies are often based on black-box neural networks. This makes them difficult to interpret and to impose desired specification constraints during learning. We present an iterative framework, MORL, for improving the learned policies using program synthesis. Concretely, we propose to use synthesis techniques to obtain a symbolic representation of the learned policy, which can then be debugged manually or automatically using program repair. After the repair step, we use behavior cloning to obtain the policy…
Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）；University of Trento（特伦托大学）
Being able to provide counterfactual interventions - sequences of actions we would have had to take for a desirable outcome to happen - is essential to explain how to change an unfavourable decision by a black-box machine learning model (e.g., being denied a loan request). Existing solutions have mainly focused on generating feasible interventions without providing explanations on their rationale. Moreover, they need to solve a separate optimization problem for each user. In this paper, we take a different approach and learn a program that outputs a sequence of explainable counterfactual actio…
Enhancing ML Models Interpretability for Credit Scoring
Dept. of Computer Science, Delft University of Technology（代尔夫特理工大学计算机科学系）；Dept. of Applied Mathematics, Delft University of Technology（代尔夫特理工大学应用数学系）；FF Quant Advisory B.V.（FF量化咨询公司）
Eindhoven University of Technology（埃因霍温理工大学）；Philips Research Eindhoven（飞利浦埃因霍温研究院）
Most state of the art decision systems based on Reinforcement Learning (RL) are data-driven black-box neural models, where it is often difficult to incorporate expert knowledge into the models or let experts review and validate the learned decision mechanisms. Knowledge-insertion and model review are important requirements in many applications involving human health and safety. One way to bridge the gap between data and knowledge driven systems is program synthesis: replacing a neural network that outputs decisions with a symbolic program generated by a neural network or by means of genetic pr…
Locally Pareto-Optimal Interpretations for Black-Box Machine Learning Models
University of California at Berkeley（加州大学伯克利分校）；Indian Institute of Technology Bombay（印度班加罗尔理工学院）；Chalmers University of Technology and University of Gothenburg（查尔姆斯理工大学和哥德堡大学）
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；Google Research（谷歌研究院）；Stanford University（斯坦福大学）；Shanghai Jiao Tong University（上海交通大学）
We consider two important aspects in understanding and editing images: modeling regular, program-like texture or patterns in 2D planes, and 3D posing of these planes in the scene. Unlike prior work on image-based program synthesis, which assumes the image contains a single visible 2D plane, we present Box Program Induction (BPI), which infers a program-like scene representation that simultaneously models repeated structure on multiple 2D planes, the 3D position and orientation of the planes, and camera parameters, all from a single image. Our model assumes a box prior, i.e., that the image cap…
