Interpretability and effectiveness are two essential and indispensable requirements for adopting machine learning methods in reality. In this paper, we propose a knowledge distillation based decision trees extension, dubbed rectified decision trees (ReDT), to explore the possibility of fulfilling those requirements simultaneously. Specifically, we extend the splitting criteria and the ending condition of the standard decision trees, which allows training with soft labels while preserving the deterministic splitting paths. We then train the ReDT based on the soft label distilled from a well-trained teacher model through a novel jackknife-based method. Accordingly, ReDT preserves the excellent interpretable nature of the decision trees while having a relatively good performance. The effectiveness of adopting soft labels instead of hard ones is also analyzed empirically and theoretically. Surprisingly, experiments indicate that the introduction of soft labels also reduces the model size compared with the standard decision trees from the aspect of the total nodes and rules, which is an unexpected gift from the `dark knowledge' distilled from the teacher model.
Generating gradients in the energy landscape using rectified linear type cost functions for efficiently solving 0/1 matrix factorization in Simulated Annealing
Fujitsu Limited（富士通株式会社）；DXR Laboratory Inc.（DXR实验室股份有限公司）；The University of Tokyo（东京大学）
On the one hand, artificial neural networks (ANNs) are commonly labelled as black-boxes, lacking interpretability; an issue that hinders human understanding of ANNs' behaviors. A need exists to generate a meaningful sequential logic of the ANN for interpreting a production process of a specific output. On the other hand, decision trees exhibit better interpretability and expressive power due to their representation language and the existence of efficient algorithms to transform the trees into rules. However, growing a decision tree based on the available data could produce larger than necessar…
Max Planck Institute for Polymer Research（马克斯·普朗克聚合物研究所）；Koc University（科奇大学）
Extracting insight from the enormous quantity of data generated from molecular simulations requires the identification of a small number of collective variables whose corresponding low-dimensional free-energy landscape retains the essential features of the underlying system. Data-driven techniques provide a systematic route to constructing this landscape, without the need for extensive a priori intuition into the relevant driving forces. In particular, autoencoders are powerful tools for dimensionality reduction, as they naturally force an information bottleneck and, thereby, a low-dimensional…
Interpretation of Crystal Energy Landscapes with Kolmogorov-Arnold Networks
Max Planck Institute for Chemical Physics of Solids（马克斯·普朗克固体化学物理研究所）；Department of Physics and Astronomy, University of Tennessee（田纳西大学物理与天文学系）；Min H. Kao Department of Electrical Engineering and Computer Science, University of Tennessee（田纳西大学高民电气工程与计算机科学系）
Despite their effective use in various fields, many aspects of neural networks are poorly understood. One important way to investigate the characteristics of neural networks is to explore the loss landscape. However, most models produce a high-dimensional non-convex landscape which is difficult to visualize. We discuss and extend existing visualization methods based on 1D- and 2D slicing with a novel method that approximates the actual loss landscape geometry by using charts with interpretable axes. Based on the assumption that observations on small neural networks can generalize to more compl…
Annual ranking of higher educational institutions (HEIs) is a global phenomenon and have significant impact on higher education landscape. Most of the HEIs pay close attention to ranking results and look forward to improving their ranks. However, maintaining a good rank and ascending in the rankings is a difficult task because it requires considerable resources, efforts and performance improvement plan. In this work, firstly, we show how exploratory data analysis (EDA) using correlation heatmaps and box plots can aid in understanding the broad trends in the ranking data. Subsequently, we prese…
Exploring Energy Landscapes for Minimal Counterfactual Explanations: Applications in Cybersecurity and Beyond
Netcompany-Intrasoft S.A.（Netcompany-Intrasoft公司）；Hellenic Mediterranean University（希腊地中海大学）；Harokopio University of Athens（雅典哈罗科皮奥大学）；University of Western Macedonia（西马其顿大学）
The Standard Interpretable Model: A general theory of interpretable machine learning to deductively design interpretable methods using Lagrangian mechanics
IBM Research (CH)（IBM研究院（瑞士））；University of Oxford (UK)（牛津大学（英国））；University of Cambridge (UK)（剑桥大学（英国））；KU Leuven (BE)（鲁汶大学（比利时））；Institute of Physics of the Czech Academy of Sciences (CZ)（捷克科学院物理研究所（捷克））
LHT: Statistically-Driven Oblique Decision Trees for Interpretable Classification
Department of Automation（自动化系）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Department of Computing Science（计算机科学系）；University of Alberta（阿尔伯塔大学）
DT-PBO: an Interpretable Tree-based Surrogate Model for Preferential Bayesian Optimization
Data Science Center of Excellence, Faculty of Military Sciences, Breda, NL（数据科学卓越中心，军事科学学院，荷兰布雷达）；NLR - Royal Netherlands Aerospace Centre, Amsterdam, NL（NLR-荷兰航空航天研究中心，荷兰阿姆斯特丹）；Departement Intelligent Systems, Tilburg University, Tilburg, NL（智能系统部门，蒂尔堡大学，荷兰蒂尔堡）
Testing Neural Networks via Bayesian-Guided Exploration of Decision Landscapes
School of Electrical Engineering and Computer Science, The University of Queensland, Australia（昆士兰大学电子工程与计算机科学学院）；College of Information and Communications Technology, Central Queensland University, Australia（中央昆士兰大学信息与通信技术学院）
Wageningen University（瓦赫宁根大学）；Oxford University（牛津大学）；CMAP, École Polytechnique（巴黎综合理工学院应用数学中心）；IRISA, University Bretagne Sud, CNRS（南布列塔尼大学、法国国家科学研究中心、IRISA）；EPFL（洛桑联邦理工学院）
Convolutional neural networks (CNN) are known to learn an image representation that captures concepts relevant to the task, but do so in an implicit way that hampers model interpretability. However, one could argue that such a representation is hidden in the neurons and can be made explicit by teaching the model to recognize semantically interpretable attributes that are present in the scene. We call such an intermediate layer a \emph{semantic bottleneck}. Once the attributes are learned, they can be re-combined to reach the final decision and provide both an accurate prediction and an explici…
Università degli Studi di Milano（米兰大学）；Politecnico di Milano（米兰理工大学）；Istituto Italiano di Tecnologia（意大利技术研究院）；Tel Aviv University（特拉维夫大学）；Google Research（谷歌研究院）；Technion（以色列理工学院）；TU Wien（维也纳技术大学）
Needles in the Landscape: Semi-Supervised Pseudolabeling for Archaeological Site Discovery under Label Scarcity
Sensors, Royal Military Academy, Brussels, Belgium AMGC (Archaeology, Environmental Changes \& Geo-Chemistry), Vrije Universiteit Brussel Max Planck Institute of Geoanthropology, Jena, Germany Shared first author Shared last author
