Isolation forest or "iForest" is an intuitive and widely used algorithm for anomaly detection that follows a simple yet effective idea: in a given data distribution, if a threshold (split point) is selected uniformly at random within the range of some variable and data points are divided according to whether they are greater or smaller than this threshold, outlier points are more likely to end up alone or in the smaller partition. The original procedure suggested the choice of variable to split and split point within a variable to be done uniformly at random at each step, but this paper shows that "clustered" diverse outliers - oftentimes a more interesting class of outliers than others - can be more easily identified by applying a non-uniformly-random choice of variables and/or thresholds. Different split guiding criteria are compared and some are found to result in significantly better outlier discrimination for certain classes of outliers.
We propose a new method, named isolation Mondrian forest (iMondrian forest), for batch and online anomaly detection. The proposed method is a novel hybrid of isolation forest and Mondrian forest which are existing methods for batch anomaly detection and online random forest, respectively. iMondrian forest takes the idea of isolation, using the depth of a node in a tree, and implements it in the Mondrian forest structure. The result is a new data structure which can accept streaming data in an online manner while being used for anomaly detection. Our experiments show that iMondrian forest mostl…
Peter the Great St.Petersburg Polytechnic University（彼得大帝圣彼得堡理工大学）
A new modification of Isolation Forest called Attention-Based Isolation Forest (ABIForest) for solving the anomaly detection problem is proposed. It incorporates the attention mechanism in the form of the Nadaraya-Watson regression into the Isolation Forest for improving solution of the anomaly detection problem. The main idea underlying the modification is to assign attention weights to each path of trees with learnable parameters depending on instances and trees themselves. The Huber's contamination model is proposed to be used for defining the attention weights and their parameters. As a re…
Function Based Isolation Forest (FuBIF): A Unifying Framework for Interpretable Isolation-Based Anomaly Detection
Brain, Mind and Computer Science, University of Padova, Italy（脑科学与计算机科学系，帕多瓦大学，意大利）；Department of Mathematics, Sapienza University of Roma, Italy（数学系，罗马萨皮恩扎大学，意大利）；Department of Information Engineering, University of Padova, Italy（信息工程系，帕多瓦大学，意大利）
Institut Pasteur（巴斯德研究所）；Université Paris Cité（巴黎城市大学）；Inserm（法国国家健康与医学研究院）；Institut de l’Audition（听觉研究所）；IHU reConnect（reConnect 医院研究所）；INRIA（法国国家信息与自动化研究所）；CEA（法国原子能和替代能源委员会）；Univ. Paris-Saclay（巴黎-萨克雷大学）；University of California Santa Barbara（加州大学圣巴巴拉分校）；Institute of Statistical Mathematics（统计数学研究所）
Anomaly Detection is an unsupervised learning task aimed at detecting anomalous behaviours with respect to historical data. In particular, multivariate Anomaly Detection has an important role in many applications thanks to the capability of summarizing the status of a complex system or observed phenomenon with a single indicator (typically called `Anomaly Score') and thanks to the unsupervised nature of the task that does not require human tagging. The Isolation Forest is one of the most commonly adopted algorithms in the field of Anomaly Detection, due to its proven effectiveness and low comp…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We present an extension to the model-free anomaly detection algorithm, Isolation Forest. This extension, named Extended Isolation Forest (EIF), resolves issues with assignment of anomaly score to given data points. We motivate the problem using heat maps for anomaly scores. These maps suffer from artifacts generated by the criteria for branching operation of the binary tree. We explain this problem in detail and demonstrate the mechanism by which it occurs visually. We then propose two different approaches for improving the situation. First we propose transforming the data randomly before crea…
From the identification of a drawback in the Isolation Forest (IF) algorithm that limits its use in the scope of anomaly detection, we propose two extensions that allow to firstly overcome the previously mention limitation and secondly to provide it with some supervised learning capability. The resulting Hybrid Isolation Forest (HIF) that we propose is first evaluated on a synthetic dataset to analyze the effect of the new meta-parameters that are introduced and verify that the addressed limitation of the IF algorithm is effectively overcame. We hen compare the two algorithms on the ISCX bench…
LTCI, Télécom Paris, Institut Polytechnique de Paris（巴黎综合理工学院电信巴黎（LTCI））
For the purpose of monitoring the behavior of complex infrastructures (e.g. aircrafts, transport or energy networks), high-rate sensors are deployed to capture multivariate data, generally unlabeled, in quasi continuous-time to detect quickly the occurrence of anomalies that may jeopardize the smooth operation of the system of interest. The statistical analysis of such massive data of functional nature raises many challenging methodological questions. The primary goal of this paper is to extend the popular Isolation Forest (IF) approach to Anomaly Detection, originally dedicated to finite dime…
Anomaly detection is concerned with identifying examples in a dataset that do not conform to the expected behaviour. While a vast amount of anomaly detection algorithms exist, little attention has been paid to explaining why these algorithms flag certain examples as anomalies. However, such an explanation could be extremely useful to anyone interpreting the algorithms' output. This paper develops a method to explain the anomaly predictions of the state-of-the-art Isolation Forest anomaly detection algorithm. The method outputs an explanation vector that captures how important each attribute of…
Preference Isolation Forest for Structure-based Anomaly Detection
Department of Electronics, Information and Bioengineering, Politecnico di Milano（电子、信息与生物工程系，米兰理工大学）；Faculty of Informatics, Università della Svizzera Italiana（信息技术学院，意大利瑞士大学）
Unsupervised anomaly detection tackles the problem of finding anomalies inside datasets without the labels availability; since data tagging is typically hard or expensive to obtain, such approaches have seen huge applicability in recent years. In this context, Isolation Forest is a popular algorithm able to define an anomaly score by means of an ensemble of peculiar trees called isolation trees. These are built using a random partitioning procedure that is extremely fast and cheap to train. However, we find that the standard algorithm might be improved in terms of memory requirements, latency …
OptIForest: Optimal Isolation Forest for Anomaly Detection
Macquarie University（麦考瑞大学）；CSIRO’s Data61（联邦科学与工业研究组织数据61研究所）；Qufu Normal University（曲阜师范大学）；Nanjing University（南京大学）；Nanjing University of Information Science and Technology（南京信息工程大学）
The detection of anomalous behaviours is an emerging need in many applications, particularly in contexts where security and reliability are critical aspects. While the definition of anomaly strictly depends on the domain framework, it is often impractical or too time consuming to obtain a fully labelled dataset. The use of unsupervised models to overcome the lack of labels often fails to catch domain specific anomalies as they rely on general definitions of outlier. This paper suggests a new active learning based approach, ALIF, to solve this problem by reducing the number of required labels a…
York University（约克大学）；iSecurity Consulting Inc（iSecurity咨询公司）；Dapasoft Inc（Dapasoft公司）
Industrial Information Technology (IT) infrastructures are often vulnerable to cyberattacks. To ensure security to the computer systems in an industrial environment, it is required to build effective intrusion detection systems to monitor the cyber-physical systems (e.g., computer networks) in the industry for malicious activities. This paper aims to build such intrusion detection systems to protect the computer networks from cyberattacks. More specifically, we propose a novel unsupervised machine learning approach that combines the K-Means algorithm with the Isolation Forest for anomaly detec…
This work briefly explores the possibility of approximating spatial distance (alternatively, similarity) between data points using the Isolation Forest method envisioned for outlier detection. The logic is similar to that of isolation: the more similar or closer two points are, the more random splits it will take to separate them. The separation depth between two points can be standardized in the same way as the isolation depth, transforming it into a distance metric that is limited in range, centered, and in compliance with the axioms of distance. This metric presents some desirable propertie…
Programmable logic controller (PLC) based industrial control systems (ICS) are used to monitor and control critical infrastructure. Integration of communication networks and an Internet of Things approach in ICS has increased ICS vulnerability to cyber-attacks. This work proposes novel unsupervised machine learning ensemble methods for anomaly detection in PLC-based ICS. The work presents two broad approaches to anomaly detection: a weighted voting ensemble approach with a learning algorithm based on coefficient of determination and a stacking-based ensemble approach using isolation forest met…
The isolation forest algorithm for outlier detection exploits a simple yet effective observation: if taking some multivariate data and making uniformly random cuts across the feature space recursively, it will take fewer such random cuts for an outlier to be left alone in a given subspace as compared to regular observations. The original idea proposed an outlier score based on the tree depth (number of random cuts) required for isolation, but experiments here show that using information about the size of the feature space taken and the number of points assigned to it can result in improved res…
In this paper, we propose DiFF-RF, an ensemble approach composed of random partitioning binary trees to detect point-wise and collective (as well as contextual) anomalies. Thanks to a distance-based paradigm used at the leaves of the trees, this semi-supervised approach solves a drawback that has been identified in the isolation forest (IF) algorithm. Moreover, taking into account the frequencies of visits in the leaves of the random trees allows to significantly improve the performance of DiFF-RF when considering the presence of collective anomalies. DiFF-RF is fairly easy to train, and excel…
Isolation-based Spherical Ensemble Representations for Tabular Anomaly Detection
Tsinghua Shenzhen International Graduate School, Tsinghua University, China（清华大学深圳国际研究生院，清华大学，中国）；School of Computing and Information Technology, Great Bay University, China（计算机与信息科技学院，Great Bay大学，中国）；School of Compuer Science, Nanjing University, China（计算机科学学院，南京大学，中国）；Computer department, Dongguan university of technology, China（计算机系，东莞科技学院，中国）；College of Computer Science and Technology, China University of Petroleum, China（计算机科学与技术学院，中国石油大学，中国）；School of IT, Deakin University, Australia（信息技术学院，德金大学，澳大利亚）
A Classifier-Based Approach to Multi-Class Anomaly Detection Applied to Astronomical Time-Series
Kavli Institute for Astrophysics and Space Research, Massachusetts Institute of Technology（麻省理工学院卡弗里天体物理与空间研究所）；Irvington High School（欧文顿高中）；Department of Physics and Astronomy, University of the Western Cape（西开普大学物理与天文学系）；South African Radio Astronomy Observatory（南非射电天文观测台）
Theoretical Investigation on Inductive Bias of Isolation Forest
National Key Laboratory for Novel Software Technology, Nanjing University, China（新型软件技术国家实验室，南京大学）；School of Artificial Intelligence, Nanjing University, China（人工智能学院，南京大学）；School of Intelligence Science and Technology, Nanjing University, China（智能科学与技术学院，南京大学）；College of Computer and Information, Hohai University, China（计算机与信息学院，河海大学）；Key Laboratory of Water Resources Big Data Technology of Ministry of Water Resources, Hohai University, China（水利部水资源大数据技术重点实验室，河海大学）
Towards eco friendly cybersecurity: machine learning based anomaly detection with carbon and energy metrics
Gannon University（甘农大学）；Western Illinois University（西伊利诺伊大学）；University of South Wales（南威尔士大学）；Washington University of Science and Technology（华盛顿科技大学）；International American University（国际美国大学）；California State University, Dominguez Hills（加州州立大学多明格斯山分校）；University of New Haven（纽黑文大学）
When digitizing a print bilingual dictionary, whether via optical character recognition or manual entry, it is inevitable that errors are introduced into the electronic version that is created. We investigate automating the process of detecting errors in an XML representation of a digitized print dictionary using a hybrid approach that combines rule-based, feature-based, and language model-based methods. We investigate combining methods and show that using random forests is a promising approach. We find that in isolation, unsupervised methods rival the performance of supervised methods. Random…
