Research in machine learning has successfully developed algorithms to build accurate classification models. However, in many real-world applications, such as healthcare, customer satisfaction, and environment protection, we want to be able to use the models to decide what actions to take.
We investigate the concept of actionability in the context of Support Vector Machines. Actionability is as important as interpretability or explainability of machine learning models, an ongoing and important research topic. Actionability is the task that gives us ways to act upon machine learning models and their predictions.
This paper finds a solution to the question of actionability on both linear and non-linear SVM models. Additionally, we introduce a way to account for weighted actions that allow for more change in certain features than others. We propose a gradient descent solution on the linear, RBF, and polynomial kernels, and we test the effectiveness of our models on both synthetic and real datasets. We are also able to explore the model's interpretability through the lens of actionability.
Harvard-MIT Health Sciences and Technology（哈佛-麻省理工学院健康科学与技术）；Massachusetts Institute of Technology（麻省理工学院）
The support vector machine (SVM) is a widely used machine learning tool for classification based on statistical learning theory. Given a set of training data, the SVM finds a hyperplane that separates two different classes of data points by the largest distance. While the standard form of SVM uses L2-norm regularization, other regularization approaches are particularly attractive for biomedical datasets where, for example, sparsity and interpretability of the classifier's coefficient values are highly desired features. Therefore, in this paper we consider different types of regularization appr…
Kempner Institute at Harvard University（哈佛大学凯默纳研究所）；University of Southern California（美国南加州大学）；Mila – Quebec AI Institute（魁北克AI研究所）；McGill University（麦吉尔大学）；Google DeepMind（谷歌DeepMind）；Tel Aviv University（特拉维夫大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Maryland（马里兰大学）；University of Oxford（牛津大学）；Northeastern University（东北大学）；Boston University（波士顿大学）
The deep network model, with the majority built on neural networks, has been proved to be a powerful framework to represent complex data for high performance machine learning. In recent years, more and more studies turn to nonneural network approaches to build diverse deep structures, and the Deep Stacking Network (DSN) model is one of such approaches that uses stacked easy-to-learn blocks to build a parameter-training-parallelizable deep network. In this paper, we propose a novel SVM-based Deep Stacking Network (SVM-DSN), which uses the DSN architecture to organize linear SVM classifiers for …
A main focus of machine learning research has been improving the generalization accuracy and efficiency of prediction models. Many models such as SVM, random forest, and deep neural nets have been proposed and achieved great success. However, what emerges as missing in many applications is actionability, i.e., the ability to turn prediction results into actions. For example, in applications such as customer relationship management, clinical prediction, and advertisement, the users need not only accurate prediction, but also actionable instructions which can transfer an input to a desirable goa…
One-class support vector machine (OC-SVM) for a long time has been one of the most effective anomaly detection methods and extensively adopted in both research as well as industrial applications. The biggest issue for OC-SVM is yet the capability to operate with large and high-dimensional datasets due to optimization complexity. Those problems might be mitigated via dimensionality reduction techniques such as manifold learning or autoencoder. However, previous work often treats representation learning and anomaly prediction separately. In this paper, we propose autoencoder based one-class supp…
The lack of interpretability often makes black-box models difficult to be applied to many practical domains. For this reason, the current work, from the black-box model input port, proposes to incorporate data-based prior information into the black-box soft-margin SVM model to enhance its interpretability. The concept and incorporation mechanism of data-based prior information are successively developed, based on which the interpretable or partly interpretable SVM optimization model is designed and then solved through handily rewriting the optimization problem as a nonlinear quadratic programm…
The University of DaNang（岘港大学）；Hue University（顺化大学）
Support vector machines (SVMs) and fuzzy rule systems are functionally equivalent under some conditions. Therefore, the learning algorithms developed in the field of support vector machines can be used to adapt the parameters of fuzzy systems. Extracting fuzzy models from support vector machines has the inherent advantage that the model does not need to determine the number of rules in advance. However, after the support vector machine learning, the complexity is usually high, and interpretability is also impaired. This paper not only proposes a complete framework for extracting interpretable …
Elastic-Net Multiple Kernel Learning: Combining Multiple Data Sources for Prediction
Centre for Artificial Intelligence, Department of Computer Science, University College London, UK（人工智能中心、计算机科学系、伦敦大学学院、英国）；Hawkes Institute, University College London, UK（霍克斯研究所、伦敦大学学院、英国）；Department of Computer Science, University College London, UK（计算机科学系、伦敦大学学院、英国）；GIGA - CRC Human Imaging, University of Liège, Belgium（GIGA- CRC 人类成像、列日大学、比利时）
In recent years, feature selection has become a challenging problem in several machine learning fields, such as classification problems. Support Vector Machine (SVM) is a well-known technique applied in classification tasks. Various methodologies have been proposed in the literature to select the most relevant features in SVM. Unfortunately, all of them either deal with the feature selection problem in the linear classification setting or propose ad-hoc approaches that are difficult to implement in practice. In contrast, we propose an embedded feature selection method based on a min-max optimi…
IIIT Kalyani（卡利亚尼印度信息技术学院）；University of Padova（帕多瓦大学）；INRIA（法国国家信息与自动化研究所）；ENS | PSL（巴黎高等师范学院（巴黎文理研究大学））
We propose a symbolic representation for support vector machines (SVMs) by means of abstract interpretation, a well-known and successful technique for designing and implementing static program analyses. We leverage this abstraction in two ways: (1) to enhance the interpretability of SVMs by deriving a novel feature importance measure, called abstract feature importance (AFI), that does not depend in any way on a given dataset of the accuracy of the SVM and is very fast to compute, and (2) for verifying stability, notably individual fairness, of SVMs and producing concrete counterexamples when …
The support vector machines (SVM) is one of the most widely used and practical optimization based classification models in machine learning because of its interpretability and flexibility to produce high quality results. However, the big data imposes a certain difficulty to the most sophisticated but relatively slow versions of SVM, namely, the nonlinear SVM. The complexity of nonlinear SVM solvers and the number of elements in the kernel matrix quadratically increases with the number of samples in training data. Therefore, both runtime and memory requirements are negatively affected. Moreover…
A Safe Screening Rule with Bi-level Optimization of $ν$ Support Vector Machine
College of Statistics and Mathematics, Yunnan University of Finance and Economics（云南财经大学统计与数学学院）；College of Science, China Agricultural University（中国农业大学理学院）
Machine Learning, Deep Learning and Data Preprocessing Techniques for Detection, Prediction, and Monitoring of Stress and Stress-related Mental Disorders: A Scoping Review
Texas A&M University（德克萨斯A&M大学）；Harvard University（哈佛大学）；University of Houston（休斯顿大学）；Ford Motor Company（福特汽车公司）
Multi-Task Learning Based on Support Vector Machines and Twin Support Vector Machines: A Comprehensive Survey
Department of Computer Science, Faculty of Mathematical Sciences, Alzahra University（计算机科学系，数学科学学院，阿扎赫拉大学）；Department of Informatics, Faculty of Science, Jan Evangelista Purkyne University（信息系，科学学院，扬·埃万杰利斯塔·普日科夫斯基大学）；Department of Computer Engineering, Science and Research Branch, IAU（计算机工程系，科学与研究分支，伊阿大学）；Department of Industrial and Systems Engineering, University of Florida（工业与系统工程系，佛罗里达大学）
When will the mist clear? On the Interpretability of Machine Learning for Medical Applications: a survey
Universidad Católica de Murcia (UCAM)（穆尔西亚天主教大学）；Universitatea Alexandru Ioan Cuza (UAIC)（亚历山德鲁·伊万·库扎大学）；IRCCS San Raffaele Pisana（圣拉斐尔皮萨纳科学住院与治疗研究所）；San Raffaele Roma Open University（罗马圣拉斐尔开放大学）
Automatic Cough Analysis for Non-Small Cell Lung Cancer Detection
Department of Electronics, Information and Bioengineering, Politecnico di Milano（电子、信息与生物工程学院，米兰理工学院）；Fondazione IRCCS Istituto Nazionale dei Tumori di Milano（米兰国家肿瘤研究所）；Department of Medicine, Section of Hematology/Oncology, University of Chicago（医学学院，血液学/肿瘤学部门，芝加哥大学）；LEARNLab, IRCCS Istituto Neurologico Carlo Besta（LEARN实验室，卡尔·贝斯塔神经病学研究所）
