Machine learning and especially deep learning have garneredtremendous popularity in recent years due to their increased performanceover other methods. The availability of large amount of data has aidedin the progress of deep learning. Nevertheless, deep learning models areopaque and often seen as black boxes. Thus, there is an inherent need tomake the models interpretable, especially so in the medical domain. Inthis work, we propose a locally interpretable method, which is inspiredby one of the recent tools that has gained a lot of interest, called localinterpretable model-agnostic explanations (LIME). LIME generates singleinstance level explanation by artificially generating a dataset aroundthe instance (by randomly sampling and using perturbations) and thentraining a local linear interpretable model. One of the major issues inLIME is the instability in the generated explanation, which is caused dueto the randomly generated dataset. Another issue in these kind of localinterpretable models is the local fidelity. We propose novel modificationsto LIME by employing an autoencoder, which serves as a better weightingfunction for the local model. We perform extensive comparisons withdifferent datasets and show that our proposed method results in bothimproved stability, as well as local fidelity.
Amirkabir University of Technology（阿米尔卡比尔理工大学）；Persian Gulf University（波斯湾大学）
Nowadays, deep neural networks are being used in many domains because of their high accuracy results. However, they are considered as "black box", means that they are not explainable for humans. On the other hand, in some tasks such as medical, economic, and self-driving cars, users want the model to be interpretable to decide if they can trust these results or not. In this work, we present a modified version of an autoencoder-based approach for local interpretability called ALIME. The ALIME itself is inspired by a famous method called Local Interpretable Model-agnostic Explanations (LIME). LI…
Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models
Technical University of Munich（慕尼黑技术大学）；Helmholtz Munich（亥姆霍兹慕尼黑）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Munich Data Science Institute（慕尼黑数据科学研究所）；University of Tübingen（图宾根大学）；University of Copenhagen（哥本哈根大学）
Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoders
School of Computing and Data Science, The University of Hong Kong（计算与数据科学学院，香港大学）；School of Data Science, The Chinese University of Hong Kong, Shenzhen（数据科学学院，香港中文大学（深圳））
Temporal Sparse Autoencoders: Leveraging the Sequential Nature of Language for Interpretability
Harvard School of Engineering and Applied Science（哈佛工程与应用科学学院）；Kempner Institute for the Study of Natural & Artificial Intelligence（自然与人工智能研究学院）；Harvard Business School（哈佛商学院）
A Multi-Level Causal Intervention Framework for Mechanistic Interpretability in Variational Autoencoders
Indian Institute of Technology, Patna（印度理工学院巴特那分校）；Rajarshi School of Management Technology, Varanasi（拉贾尔希管理技术学院瓦拉纳西分校）；Jaypee Institute of Information Technology, Noida（杰皮信息技术学院诺伊达分校）
Imperial College London（帝国理工学院）；The Alan Turing Institute（艾伦·图灵研究所）；Delft University of Technology（代尔夫特理工大学）
The dynamics of a turbulent flow tend to occupy only a portion of the phase space at a statistically stationary regime. From a dynamical systems point of view, this portion is the attractor. The knowledge of the turbulent attractor is useful for two purposes, at least: (i) We can gain physical insight into turbulence (what is the shape and geometry of the attractor?), and (ii) it provides the minimal number of degrees of freedom to accurately describe the turbulent dynamics. Autoencoders enable the computation of an optimal latent space, which is a low-order representation of the dynamics. If …
Disentangled Latent Spaces for Reduced Order Models using Deterministic Autoencoders
Institute for Fluid Dynamics and Ship Theory, Hamburg University of Technology（汉堡工业大学流体动力学与船舶理论研究所）；Institute of Mathematics, Hamburg University of Technology（汉堡工业大学数学研究所）
Posterior-Calibrated Causal Circuits in Variational Autoencoders: Why Image-Domain Interpretability Fails on Tabular Data
Indian Institute of Technology, Patna（印度理工学院巴特那分校）；Rajarshi School of Management Technology, Varanasi（拉贾尔希管理技术学院瓦拉纳西分校）；Jaypee Institute of Information Technology, Noida（杰皮信息技术学院诺伊达分校）
School of Information Systems, Science and Engineering Faculty of Queensland University of Technology（昆士兰科技大学科学与工程学院信息系统学院）；Department of Computer Science and Engineering, Instituto Superior Técnico / INESC-ID, University of Lisbon（里斯本大学高等技术学院/INESC-ID计算机科学与工程系）
This paper explores interpretability techniques for two of the most successful learning algorithms in medical decision-making literature: deep neural networks and random forests. We applied these algorithms in a real-world medical dataset containing information about patients with cancer, where we learn models that try to predict the type of cancer of the patient, given their set of medical activity records. We explored different algorithms based on neural network architectures using long short term deep neural networks, and random forests. Since there is a growing need to provide decision-mak…
University of Applied Sciences and Arts Western Switzerland (HES-SO)（瑞士西部应用科学与艺术大学（HES-SO））
Integrating physics models within machine learning models holds considerable promise toward learning robust models with improved interpretability and abilities to extrapolate. In this work, we focus on the integration of incomplete physics models into deep generative models. In particular, we introduce an architecture of variational autoencoders (VAEs) in which a part of the latent space is grounded by physics. A key technical challenge is to strike a balance between the incomplete physics and trainable components such as neural networks for ensuring that the physics part is used in a meaningf…
Mechanistic Interpretability with Sparse Autoencoder Neural Operators
University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute (Amii)（阿尔伯塔机器智能研究所（Amii））；California Institute of Technology (Caltech)（加州理工学院（Caltech））
Disentangling Dense Embeddings with Sparse Autoencoders
The Australian National University（澳大利亚国立大学）；Stanford University（斯坦福大学）；Columbia University（哥伦比亚大学）；Space Telescope Science Institute（空间望远镜科学研究所）；Johns Hopkins University（约翰斯·霍普金斯大学）
Recently, deep learning becomes the main focus of machine learning research and has greatly impacted many important fields. However, deep learning is criticized for lack of interpretability. As a successful unsupervised model in deep learning, the autoencoder embraces a wide spectrum of applications, yet it suffers from the model opaqueness as well. In this paper, we propose a new type of convolutional autoencoders, termed as Soft Autoencoder (Soft-AE), in which the activation functions of encoding layers are implemented with adaptable soft-thresholding units while decoding layers are realized…
Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders
BrainCapture；DTU Health Tech（技术大学丹麦健康技术）；DTU Compute（技术大学丹麦计算）；Department of Biomedical Data Science（生物医学数据科学系）；Department of Computer Science（计算机科学系）；Seer Medical（Seer医疗）；Filadelfia Epilepsy Hospital（菲拉德尔菲亚癫痫医院）；University Hospital of Copenhagen（哥本哈根大学医院）
This paper introduces the Descriptive Variational Autoencoder (DVAE), an unsupervised and end-to-end trainable neural network for predicting vehicle trajectories that provides partial interpretability. The novel approach is based on the architecture and objective of common variational autoencoders. By introducing expert knowledge within the decoder part of the autoencoder, the encoder learns to extract latent parameters that provide a graspable meaning in human terms. Such an interpretable latent space enables the validation by expert defined rule sets. The evaluation of the DVAE is performed …
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Massachusetts General Hospital（麻省总医院）；Northeastern University（东北大学）
Inspired by complexity and diversity of biological neurons, our group proposed quadratic neurons by replacing the inner product in current artificial neurons with a quadratic operation on input data, thereby enhancing the capability of an individual neuron. Along this direction, we are motivated to evaluate the power of quadratic neurons in popular network architectures, simulating human-like learning in the form of quadratic-neuron-based deep learning. Our prior theoretical studies have shown important merits of quadratic neurons and networks in representation, efficiency, and interpretabilit…
SplInterp: Improving our Understanding and Training of Sparse Autoencoders
School of Mathematics University of Birmingham（数学学院英国伯明翰大学）；Ideami Studios（Ideami工作室）；Department of Mathematics and Statistics University of Limerick（数学与统计学学院英国利默里克大学）；XRAI Inc.（XRAI公司）；Department of Computer Science Brown University（计算机科学系布朗大学）
Ikerlan Technology Research Centre（伊克尔兰技术研究中心）；Basque Research and Technology Alliance (BRTA)（巴斯克研究与技术联盟）；Mondragon University（蒙德拉贡大学）
Deep Learning has become a very valuable tool in different fields, and no one doubts the learning capacity of these models. Nevertheless, since Deep Learning models are often seen as black boxes due to their lack of interpretability, there is a general mistrust in their decision-making process. To find a balance between effectiveness and interpretability, Explainable Artificial Intelligence (XAI) is gaining popularity in recent years, and some of the methods within this area are used to generate counterfactual explanations. The process of generating these explanations generally consists of sol…
Technical University of Munich（慕尼黑工业大学）；Klinikum Rechts der Isar (Technical University of Munich)（慕尼黑工业大学伊萨尔河右岸医院）；University of Munich（慕尼黑大学）；Klinikum der Universität München（慕尼黑大学医院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Vertebral fractures are a consequence of osteoporosis, with significant health implications for affected patients. Unfortunately, grading their severity using CT exams is hard and subjective, motivating automated grading methods. However, current approaches are hindered by imbalance and scarcity of data and a lack of interpretability. To address these challenges, this paper proposes a novel approach that leverages unlabelled data to train a generative Diffusion Autoencoder (DAE) model as an unsupervised feature extractor. We model fracture grading as a continuous regression, which is more refl…
