The vast majority of research on explainability focuses on post-explainability rather than explainable modeling. Namely, an explanation model is derived to explain a complex black box model built with the sole purpose of achieving the highest performance possible. In part, this trend might be driven by the misconception that there is a trade-off between explainability and accuracy. Furthermore, the consequential work on Shapely values, grounded in game theory, has also contributed to a new wave of post-explainability research on better approximations for various machine learning models, including deep learning models. We propose a new architecture that inherently produces explainable predictions in the form of additive feature attributions. Our approach learns a graph representation for each record in the dataset. Attribute centric features are then derived from the graph and fed into a contribution deep set model to produce the final predictions. We show that our explainable model attains the same level of performance as black box models. Finally, we provide an augmented model training approach that leverages the missingness property and yields high levels of consistency (as required for the Shapely values) without loss of accuracy.
Institute for Informatics and Telematics (IIT), National Research Council of Italy (CNR)（意大利国家研究委员会（CNR）信息学与电信研究所（IIT））
Nowadays Artificial Intelligence (AI) has become a fundamental component of healthcare applications, both clinical and remote, but the best performing AI systems are often too complex to be self-explaining. Explainable AI (XAI) techniques are defined to unveil the reasoning behind the system's predictions and decisions, and they become even more critical when dealing with sensitive and personal health data. It is worth noting that XAI has not gathered the same attention across different research areas and data types, especially in healthcare. In particular, many clinical and remote health appl…
Although modern machine learning and deep learning methods allow for complex and in-depth data analytics, the predictive models generated by these methods are often highly complex, and lack transparency. Explainable AI (XAI) methods are used to improve the interpretability of these complex models, and in doing so improve transparency. However, the inherent fitness of these explainable methods can be hard to evaluate. In particular, methods to evaluate the fidelity of the explanation to the underlying black box require further development, especially for tabular data. In this paper, we (a) prop…
What's Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Models
Leibniz Institute for Prevention Research & Epidemiology – BIPS（莱比锡预防研究与流行病学研究所 – BIPS）；Faculty of Mathematics and Computer Science, University of Bremen（数学与计算机科学学院，不莱梅大学）；Norwegian Computing Center（挪威计算中心）
FlagGAM: Rule-Basis Generalized Additive Models for Explainable Tabular Prediction
EECS Department, Massachusetts Institute of Technology, Cambridge, MA, USA（麻省理工学院电子工程与计算机科学系）；Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA（麻省理工学院斯隆管理学院）
HyConEx: Hypernetwork classifier with counterfactual explanations for tabular data
Faculty of Mathematics and Computer Science, Jagiellonian University（数学与计算机科学学院，杰兹维日大学）；Wrocław University of Science and Technology（沃拉什大学技术学院）；IDEAS Research Institute（IDEAS研究所）
The University of Arizona（亚利桑那大学）；Virginia Tech（弗吉尼亚理工大学）；University of Cincinnati（辛辛那提大学）
A longstanding challenge surrounding deep learning algorithms is unpacking and understanding how they make their decisions. Explainable Artificial Intelligence (XAI) offers methods to provide explanations of internal functions of algorithms and reasons behind their decisions in ways that are interpretable and understandable to human users. . Numerous XAI approaches have been developed thus far, and a comparative analysis of these strategies seems necessary to discern their relevance to clinical prediction models. To this end, we first implemented two prediction models for short- and long-term …
An Explainable Multi-Task Similarity Measure: Integrating Accumulated Local Effects and Weighted Fréchet Distance
Department of Computer Science, University of Alcala（阿尔卡拉大学计算机科学系）；Faculty of Law, Business and Government, Universidad Francisco de Vitoria（弗朗西斯科·维多利亚大学法学院、商业与政府学院）
Explainable Dual-Attention Tabular Transformer for Soil Electrical Resistivity Prediction: A Decision Support Framework for High-Voltage Substation Construction
King Mongkut’s University of technology Thonburi（泰国国王科技大学）
本研究提出一种用于预测土壤电阻率的双注意力 Transformer 架构，结合粒子群优化调参，在保持高预测精度的同时通过 SHAP 值提供可解释性，并开发了辅助高压变电站建设的 Web 决策支持框架。
We propose TabTransformer, a novel deep tabular data modeling architecture for supervised and semi-supervised learning. The TabTransformer is built upon self-attention based Transformers. The Transformer layers transform the embeddings of categorical features into robust contextual embeddings to achieve higher prediction accuracy. Through extensive experiments on fifteen publicly available datasets, we show that the TabTransformer outperforms the state-of-the-art deep learning methods for tabular data by at least 1.0% on mean AUC, and matches the performance of tree-based ensemble models. Furt…
There is an increasing interest in the application of deep learning architectures to tabular data. One of the state-of-the-art solutions is TabTransformer which incorporates an attention mechanism to better track relationships between categorical features and then makes use of a standard MLP to output its final logits. In this paper we propose multiple modifications to the original TabTransformer performing better on binary classification tasks for three separate datasets with more than 1% AUROC gains. Inspired by gated MLP, linear projections are implemented in the MLP block and multiple acti…
ExBigBang: A Dynamic Approach for Explainable Persona Classification through Contextualized Hybrid Transformer Analysis
School of Computing, Macquarie University（麦考瑞大学计算机学院）；Department of Computer Engineering and Information Technology, Shiraz University of Technology（谢里兹技术大学计算机工程与信息科技系）；John Walsh Centre for Rehabilitation Research, Kolling Institute, University of Sydney（悉尼大学约翰·沃什康复研究中心）；College of Engineering and Information Technology, University of Dubai（迪拜大学工程与信息科技学院）
IBM Research（IBM研究院）；T.J Watson Research Center（T.J沃森研究中心）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
Tabular datasets are ubiquitous in data science applications. Given their importance, it seems natural to apply state-of-the-art deep learning algorithms in order to fully unlock their potential. Here we propose neural network models that represent tabular time series that can optionally leverage their hierarchical structure. This results in two architectures for tabular time series: one for learning representations that is analogous to BERT and can be pre-trained end-to-end and used in downstream tasks, and one that is akin to GPT and can be used for generation of realistic synthetic tabular …
Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Beijing Jiaotong University（北京交通大学）
Tabular data is the foundation of the information age and has been extensively studied. Recent studies show that neural-based models are effective in learning contextual representation for tabular data. The learning of an effective contextual representation requires meaningful features and a large amount of data. However, current methods often fail to properly learn a contextual representation from the features without semantic information. In addition, it's intractable to enlarge the training set through mixed tabular datasets due to the difference between datasets. To address these problems,…
Algorithmic approaches to interpreting machine learning models have proliferated in recent years. We carry out human subject tests that are the first of their kind to isolate the effect of algorithmic explanations on a key aspect of model interpretability, simulatability, while avoiding important confounding experimental factors. A model is simulatable when a person can predict its behavior on new inputs. Through two kinds of simulation tests involving text and tabular data, we evaluate five explanations methods: (1) LIME, (2) Anchor, (3) Decision Boundary, (4) a Prototype model, and (5) a Com…
The University of Tokyo（东京大学）；Preferred Networks（首选网络）
Missing value imputation in machine learning is the task of estimating the missing values in the dataset accurately using available information. In this task, several deep generative modeling methods have been proposed and demonstrated their usefulness, e.g., generative adversarial imputation networks. Recently, diffusion models have gained popularity because of their effectiveness in the generative modeling task in images, texts, audio, etc. To our knowledge, less attention has been paid to the investigation of the effectiveness of diffusion models for missing value imputation in tabular data…
MIT LIDS（麻省理工学院信息与决策系统实验室）；MRC-BSU, University of Cambridge（剑桥大学MRC生物统计组）；Universidad Rey Juan Carlos（胡安·卡洛斯国王大学）
Modeling the probability distribution of rows in tabular data and generating realistic synthetic data is a non-trivial task. Tabular data usually contains a mix of discrete and continuous columns. Continuous columns may have multiple modes whereas discrete columns are sometimes imbalanced making the modeling difficult. Existing statistical and deep neural network models fail to properly model this type of data. We design TGAN, which uses a conditional generative adversarial network to address these challenges. To aid in a fair and thorough comparison, we design a benchmark with 7 simulated and…
Rethinking Distribution Shifts: Empirical Analysis and Modeling for Tabular Data
Department of Industrial Engineering and Operations Research（工业工程与运筹学系）；Department of Computer Science and Technology（计算机科学与技术系）；Decision, Risk, and Operations Division（决策、风险与运营部）；Columbia University（哥伦比亚大学）；Tsinghua University（清华大学）
Deep visual models have widespread applications in high-stake domains. Hence, their black-box nature is currently attracting a large interest of the research community. We present the first survey in Explainable AI that focuses on the methods and metrics for interpreting deep visual models. Covering the landmark contributions along the state-of-the-art, we not only provide a taxonomic organization of the existing techniques, but also excavate a range of evaluation metrics and collate them as measures of different properties of model explanations. Along the insightful discussion on the current …
We introduce OmniXAI (short for Omni eXplainable AI), an open-source Python library of eXplainable AI (XAI), which offers omni-way explainable AI capabilities and various interpretable machine learning techniques to address the pain points of understanding and interpreting the decisions made by machine learning (ML) in practice. OmniXAI aims to be a one-stop comprehensive library that makes explainable AI easy for data scientists, ML researchers and practitioners who need explanation for various types of data, models and explanation methods at different stages of ML process (data exploration, …
Tabular data is a common form of organizing data. Multiple models are available to generate synthetic tabular datasets where observations are independent, but few have the ability to produce relational datasets. Modeling relational data is challenging as it requires modeling both a "parent" table and its relationships across tables. We introduce REaLTabFormer (Realistic Relational and Tabular Transformer), a tabular and relational synthetic data generation model. It first creates a parent table using an autoregressive GPT-2 model, then generates the relational dataset conditioned on the parent…
Structural Equation-VAE: Disentangled Latent Representations for Tabular Data
Department of Politics and Public Administration（政治与公共行政系）；The University of Hong Kong（香港大学）；School of Computer Science（计算机科学系）；Carnegie Mellon University（卡内基梅隆大学）；Department of Applied Social Sciences（应用社会科学系）；The Hong Kong Polytechnic University（香港理工大学）
Airt Research（Airt研究中心）；Algebra University College（代数大学学院）
Time-series classification is one of the most frequently performed tasks in industrial data science, and one of the most widely used data representation in the industrial setting is tabular representation. In this work, we propose a novel scalable architecture for learning representations from tabular time-series data and subsequently performing downstream tasks such as time-series classification. The representation learning framework is end-to-end, akin to bidirectional encoder representations from transformers (BERT) in language modeling, however, we introduce novel masking technique suitabl…
