Off-policy evaluation in reinforcement learning offers the chance of using observational data to improve future outcomes in domains such as healthcare and education, but safe deployment in high stakes settings requires ways of assessing its validity. Traditional measures such as confidence intervals may be insufficient due to noise, limited data and confounding. In this paper we develop a method that could serve as a hybrid human-AI system, to enable human experts to analyze the validity of policy evaluation estimates. This is accomplished by highlighting observations in the data whose removal will have a large effect on the OPE estimate, and formulating a set of rules for choosing which ones to present to domain experts for validation. We develop methods to compute exactly the influence functions for fitted Q-evaluation with two different function classes: kernel-based and linear least squares, as well as importance sampling methods. Experiments on medical simulations and real-world intensive care unit data demonstrate that our method can be used to identify limitations in the evaluation process and make evaluation more robust.
University of Illinois Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校）；Tsinghua University（清华大学）；Tencent（腾讯）；Zhejiang University（浙江大学）
Distant supervision (DS) is a strong way to expand the datasets for enhancing relation extraction (RE) models but often suffers from high label noise. Current works based on attention, reinforcement learning, or GAN are black-box models so they neither provide meaningful interpretation of sample selection in DS nor stability on different domains. On the contrary, this work proposes a novel model-agnostic instance sampling method for DS by influence function (IF), namely REIF. Our method identifies favorable/unfavorable instances in the bag based on IF, then does dynamic instance sampling. We d…
University of New Mexico（新墨西哥大学）；Verint Intelligent Self-Service（Verint智能自助服务）
In (Yang et al. 2016), a hierarchical attention network (HAN) is created for document classification. The attention layer can be used to visualize text influential in classifying the document, thereby explaining the model's prediction. We successfully applied HAN to a sequential analysis task in the form of real-time monitoring of turn taking in conversations. However, we discovered instances where the attention weights were uniform at the stopping point (indicating all turns were equivalently influential to the classifier), preventing meaningful visualization for real-time human review or cla…
Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the RL problems, but often remains unexamined and uninterpreted. To our knowledge, this work develops the first mimic learning framework for Q functions in DRL. We introduce Linear Model U-trees (LMUTs) to approximate neural network predictions. An LMUT is learned using a novel on-l…
Trinity College Dublin（都柏林三一学院）；TU Dublin（都柏林理工大学）；Accenture Labs（埃森哲实验室）
Despite the widespread use of Knowledge Graph Embeddings (KGE), little is known about the security vulnerabilities that might disrupt their intended behaviour. We study data poisoning attacks against KGE models for link prediction. These attacks craft adversarial additions or deletions at training time to cause model failure at test time. To select adversarial deletions, we propose to use the model-agnostic instance attribution methods from Interpretable Machine Learning, which identify the training instances that are most influential to a neural model's predictions on test instances. We use t…
Technische Universität Berlin（柏林工业大学）；Max Planck Institute for Human Cognitive and Brain Sciences（马克斯·普朗克人类认知与脑科学研究所）；Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；National Institute of Mental Health（国家精神卫生研究所）；Naval Postgraduate School（海军研究生院）
A central goal in the cognitive sciences is the development of numerical models for mental representations of object concepts. This paper introduces Variational Interpretable Concept Embeddings (VICE), an approximate Bayesian method for embedding object concepts in a vector space using data collected from humans in a triplet odd-one-out task. VICE uses variational inference to obtain sparse, non-negative representations of object concepts with uncertainty estimates for the embedding values. These estimates are used to automatically select the dimensions that best explain the data. We derive a …
In this paper, we propose a deep learning model to forecast the range of increase in COVID-19 infected cases in future days and we present a novel method to compute equidimensional representations of multivariate time series and multivariate spatial time series data. Using this novel method, the proposed model can both take in a large number of heterogeneous features, such as census data, intra-county mobility, inter-county mobility, social distancing data, past growth of infection, among others, and learn complex interactions between these features. Using data collected from various sources, …
Optimal transport (OT) is a widely used technique for distribution alignment, with applications throughout the machine learning, graphics, and vision communities. Without any additional structural assumptions on trans-port, however, OT can be fragile to outliers or noise, especially in high dimensions. Here, we introduce a new form of structured OT that simultaneously learns low-dimensional structure in data while leveraging this structure to solve the alignment task. Compared with OT, the resulting transport plan has better structural interpretability, highlighting the connections between ind…
The medical domain is often subject to information overload. The digitization of healthcare, constant updates to online medical repositories, and increasing availability of biomedical datasets make it challenging to effectively analyze the data. This creates additional work for medical professionals who are heavily dependent on medical data to complete their research and consult their patients. This paper aims to show how different text highlighting techniques can capture relevant medical context. This would reduce the doctors' cognitive load and response time to patients by facilitating them …
Interpretable Machine Learning for Survival Analysis
Leibniz Institute for Prevention Research and Epidemiology – BIPS（莱比锡预防研究与流行病学研究所）；Faculty of Mathematics and Computer Science, University of Bremen（不莱梅大学数学与计算机科学学院）；Faculty of Mathematics and Information Science, Warsaw University of Technology（华沙技术大学数学与信息科学学院）；Faculty of Mathematics, Informatics and Mechanics, University of Warsaw（华沙大学数学、信息学与力学学院）；Department of Public Health, University of Copenhagen（哥本哈根大学公共卫生系）
IBM Research Europe（IBM欧洲研究院）；University of Applied Sciences of Western Switzerland（瑞士西部应用科学大学）；ETH Zürich（苏黎世联邦理工学院）
Interpretability of deep learning is widely used to evaluate the reliability of medical imaging models and reduce the risks of inaccurate patient recommendations. For models exceeding human performance, e.g. predicting RNA structure from microscopy images, interpretable modelling can be further used to uncover highly non-trivial patterns which are otherwise imperceptible to the human eye. We show that interpretability can reveal connections between the microscopic appearance of cancer tissue and its gene expression profiling. While exhaustive profiling of all genes from the histology images is…
Predicting Postoperative Stroke in Elderly SICU Patients: An Interpretable Machine Learning Model Using MIMIC Data
Department of Industrial and Systems Engineering, University of Southern California（1 工业与系统工程系，南加州大学）；Colorectal Research Center, Iran University of Medical Sciences（2 肠道癌研究中心，伊朗医学科学大学）；Department of Health Science, California State University, Long Beach (CSULB)（3 健康科学系，长滩加州州立大学（CSULB））；Department of Industrial and Manufacturing Engineering, California State Polytechnic University, Pomona（4 工业与制造工程系，庞纳加州州立理工大学）
Max Planck Institute for Polymer Research（马克斯·普朗克聚合物研究所）；Koc University（科奇大学）
Extracting insight from the enormous quantity of data generated from molecular simulations requires the identification of a small number of collective variables whose corresponding low-dimensional free-energy landscape retains the essential features of the underlying system. Data-driven techniques provide a systematic route to constructing this landscape, without the need for extensive a priori intuition into the relevant driving forces. In particular, autoencoders are powerful tools for dimensionality reduction, as they naturally force an information bottleneck and, thereby, a low-dimensional…
Interpretable Machine Learning for Cognitive Aging: Handling Missing Data and Uncovering Social Determinant
University of Texas Rio Grande Valley（德克萨斯大学里奥格兰德河谷分校）；Shandong Mental Health Center（山东省精神卫生中心）；Georgia Institute of Technology（佐治亚理工学院）；University of Texas Austin（德克萨斯大学奥斯汀分校）；Kennesaw State University（肯尼索州立大学）
Recently, the ever-growing demand for privacy-oriented machine learning has motivated researchers to develop federated and decentralized learning techniques, allowing individual clients to train models collaboratively without disclosing their private datasets. However, widespread adoption has been limited in domains relying on high levels of user trust, where assessment of data compatibility is essential. In this work, we define and address low interoperability induced by underlying client data inconsistencies in federated learning for tabular data. The proposed method, iFedAvg, builds on fede…
Insight Centre for Data Analytics（洞察数据分析中心）；University College Dublin（都柏林大学学院）
The time series classification literature has expanded rapidly over the last decade, with many new classification approaches published each year. Prior research has mostly focused on improving the accuracy and efficiency of classifiers, with interpretability being somewhat neglected. This aspect of classifiers has become critical for many application domains and the introduction of the EU GDPR legislation in 2018 is likely to further emphasize the importance of interpretable learning algorithms. Currently, state-of-the-art classification accuracy is achieved with very complex models based on l…
Can Data-Driven Dynamics Reveal Hidden Physics? There Is A Need for Interpretable Neural Operators
Department of Applied Mathematics and Statistics, Stony Brook University（应用数学与统计学系，石英布鲁克大学）；Department of Computer Science, Stony Brook University（计算机科学系，石英布鲁克大学）
