A Unifying Review of Deep and Shallow Anomaly Detection
Technische Universität Berlin（柏林工业大学）；Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technische Universität Kaiserslautern（凯泽斯劳滕工业大学）；Oregon State University（俄勒冈州立大学）；Google Research（谷歌研究院）；Korea University（高丽大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）
Lukas Ruff, Jacob R. Kauffmann, Robert A. Vandermeulen, Grégoire Montavon, Wojciech Samek, Marius Kloft, Thomas G. Dietterich, Klaus-Robert Müller
备注：
40 pages; accepted for publication in the Proceedings of the IEEE;
Deep learning approaches to anomaly detection have recently improved the state of the art in detection performance on complex datasets such as large collections of images or text. These results have sparked a renewed interest in the anomaly detection problem and led to the introduction of a great variety of new methods. With the emergence of numerous such methods, including approaches based on generative models, one-class classification, and reconstruction, there is a growing need to bring methods of this field into a systematic and unified perspective. In this review we aim to identify the common underlying principles as well as the assumptions that are often made implicitly by various methods. In particular, we draw connections between classic 'shallow' and novel deep approaches and show how this relation might cross-fertilize or extend both directions. We further provide an empirical assessment of major existing methods that is enriched by the use of recent explainability techniques, and present specific worked-through examples together with practical advice. Finally, we outline critical open challenges and identify specific paths for future research in anomaly detection.
Function Based Isolation Forest (FuBIF): A Unifying Framework for Interpretable Isolation-Based Anomaly Detection
Brain, Mind and Computer Science, University of Padova, Italy（脑科学与计算机科学系，帕多瓦大学，意大利）；Department of Mathematics, Sapienza University of Roma, Italy（数学系，罗马萨皮恩扎大学，意大利）；Department of Information Engineering, University of Padova, Italy（信息工程系，帕多瓦大学，意大利）
Tsinghua University（清华大学）；Hong Kong University of Science and Technology（香港科技大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Singapore University of Technology & Design（新加坡科技设计大学）；Bonn-Aachen International Center for Information Technology（波恩-亚琛信息技术国际中心）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Technical University of Kaiserslautern（凯泽斯劳滕工业大学）；Technical University of Berlin（柏林工业大学）；Korea University（高丽大学）
Deep approaches to anomaly detection have recently shown promising results over shallow methods on large and complex datasets. Typically anomaly detection is treated as an unsupervised learning problem. In practice however, one may have---in addition to a large set of unlabeled samples---access to a small pool of labeled samples, e.g. a subset verified by some domain expert as being normal or anomalous. Semi-supervised approaches to anomaly detection aim to utilize such labeled samples, but most proposed methods are limited to merely including labeled normal samples. Only a few methods take ad…
Anomaly detection, finding patterns that substantially deviate from those seen previously, is one of the fundamental problems of artificial intelligence. Recently, classification-based methods were shown to achieve superior results on this task. In this work, we present a unifying view and propose an open-set method, GOAD, to relax current generalization assumptions. Furthermore, we extend the applicability of transformation-based methods to non-image data using random affine transformations. Our method is shown to obtain state-of-the-art accuracy and is applicable to broad data types. The str…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Harvard University（哈佛大学）
In this paper, we propose Normality-Calibrated Autoencoder (NCAE), which can boost anomaly detection performance on the contaminated datasets without any prior information or explicit abnormal samples in the training phase. The NCAE adversarially generates high confident normal samples from a latent space having low entropy and leverages them to predict abnormal samples in a training dataset. NCAE is trained to minimise reconstruction errors in uncontaminated samples and maximise reconstruction errors in contaminated samples. The experimental results demonstrate that our method outperforms sha…
In recent years, there has been increased research interest in detecting anomalies in temporal streaming data. A variety of algorithms have been developed in the data mining community, which can be divided into two categories (i.e., general and ad hoc). In most cases, general approaches assume the one-size-fits-all solution model where a single anomaly detector can detect all anomalies in any domain. To date, there exists no single general method that has been shown to outperform the others across different anomaly types, use cases and datasets. In this paper, we propose SAFARI, a general fram…
University of Sydney（悉尼大学）；Capital Markets Co-operative Research Centre (CMCRC)（资本市场合作研究中心）；Data61/CSIRO（澳大利亚联邦科学与工业研究组织数据61中心）；Australian National University（澳大利亚国立大学）；Qatar Computing Research Institute (QCRI)（卡塔尔计算研究所）；HBKU（哈马德·本·哈利法大学）
We propose a one-class neural network (OC-NN) model to detect anomalies in complex data sets. OC-NN combines the ability of deep networks to extract a progressively rich representation of data with the one-class objective of creating a tight envelope around normal data. The OC-NN approach breaks new ground for the following crucial reason: data representation in the hidden layer is driven by the OC-NN objective and is thus customized for anomaly detection. This is a departure from other approaches which use a hybrid approach of learning deep features using an autoencoder and then feeding the f…
TU Kaiserslautern（凯泽斯劳滕工业大学）；Fraunhofer ITWM（弗劳恩霍夫工业数学研究所）
In this paper, we introduce the Vehicle Claims dataset, consisting of fraudulent insurance claims for automotive repairs. The data belongs to the more broad category of Auditing data, which includes also Journals and Network Intrusion data. Insurance claim data are distinctively different from other auditing data (such as network intrusion data) in their high number of categorical attributes. We tackle the common problem of missing benchmark datasets for anomaly detection: datasets are mostly confidential, and the public tabular datasets do not contain relevant and sufficient categorical attri…
Monash University（莫纳什大学）；Shanghai Jiao Tong University（上海交通大学）；Max Planck Institute for Mathematics in Sciences（马克斯·普朗克数学与科学研究所）；Beijing Jiaotong University（北京交通大学）；Northwestern Polytechnical University（西北工业大学）；Guangxi University（广西大学）
Detecting anomalies for dynamic graphs has drawn increasing attention due to their wide applications in social networks, e-commerce, and cybersecurity. Recent deep learning-based approaches have shown promising results over shallow methods. However, they fail to address two core challenges of anomaly detection in dynamic graphs: the lack of informative encoding for unattributed nodes and the difficulty of learning discriminate knowledge from coupled spatial-temporal dynamic graphs. To overcome these challenges, in this paper, we present a novel Transformer-based Anomaly Detection framework for…
Anomaly detection is a challenging problem in machine learning, and is even more so when dealing with instances that are captured in low-level, raw data representations without a well-behaved set of engineered features. The Radial Basis Function Data Descriptor (RBFDD) network is an effective solution for anomaly detection, however, it is a shallow model that does not deal effectively with raw data representations. This paper investigates approaches to modifying the RBFDD network to transform it into a deep one-class classifier suitable for anomaly detection problems with low-level raw data re…
Anomaly detection for indoor air quality (IAQ) data has become an important area of research as the quality of air is closely related to human health and well-being. However, traditional statistics and shallow machine learning-based approaches in anomaly detection in the IAQ area could not detect anomalies involving the observation of correlations across several data points (i.e., often referred to as long-term dependences). We propose a hybrid deep learning model that combines LSTM with Autoencoder for anomaly detection tasks in IAQ to address this issue. In our approach, the LSTM network is …
School of Computer Science and Engineering, The Hebrew University of Jerusalem（耶路撒冷希伯来大学计算机科学与工程学院）
Detecting anomalous time series is key for scientific, medical and industrial tasks, but is challenging due to its inherent unsupervised nature. In recent years, progress has been made on this task by learning increasingly more complex features, often using deep neural networks. In this work, we argue that shallow features suffice when combined with distribution distance measures. Our approach models each time series as a high dimensional empirical distribution of features, where each time-point constitutes a single sample. Modeling the distance between a test time series and the normal traini…
Humans can easily detect a defect (anomaly) because it is different or salient when compared to the surface it resides on. Today, manual human visual inspection is still the norm because it is difficult to automate anomaly detection. Neural networks are a useful tool that can teach a machine to find defects. However, they require a lot of training examples to learn what a defect is and it is tedious and expensive to get these samples. We tackle the problem of teaching a network with a low number of training samples with a system we call AnoNet. AnoNet's architecture is similar to CompactCNN wi…
Court of LLMs: Evidence-Augmented Generation via Multi-LLM Collaboration for Text-Attributed Graph Anomaly Detection
School of Computer Science and Technology, Xi'an Jiaotong University（计算机科学与技术学院，西安交通大学）；Department of Electrical and Computer Engineering, University of Virginia（电气与计算机工程系，弗吉尼亚大学）；Saw Swee Hock School of Public Health, National University of Singapore（Saw Swee Hock 公共卫生学院，新加坡国立大学）；China Telecom Corporation Ltd. Shaanxi Branch（中国电信股份有限公司陕西分公司）；School of Distance Education, Xi'an Jiaotong University（继续教育学院，西安交通大学）
School of Computing, National University of Singapore（新加坡国立大学计算学院）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）
Many well-established anomaly detection methods use the distance of a sample to those in its local neighbourhood: so-called `local outlier methods', such as LOF and DBSCAN. They are popular for their simple principles and strong performance on unstructured, feature-based data that is commonplace in many practical applications. However, they cannot learn to adapt for a particular set of data due to their lack of trainable parameters. In this paper, we begin by unifying local outlier methods by showing that they are particular cases of the more general message passing framework used in graph neu…
ResAD: A Simple Framework for Class Generalizable Anomaly Detection
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；China Pacific Insurance (Group) Co., Ltd.（中国太平洋保险（集团）股份有限公司）；MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能研究院、教育部人工智能重点实验室）
LMU Munich（慕尼黑大学）；Stadtwerke München GmbH（慕尼黑市政公用事业有限公司）
In this work, we present a general procedure for acoustic leak detection in water networks that satisfies multiple real-world constraints such as energy efficiency and ease of deployment. Based on recordings from seven contact microphones attached to the water supply network of a municipal suburb, we trained several shallow and deep anomaly detection models. Inspired by how human experts detect leaks using electronic sounding-sticks, we use these models to repeatedly listen for leaks over a predefined decision horizon. This way we avoid constant monitoring of the system. While we found the det…
La Trobe University（乐卓博大学）；Monash University（蒙纳士大学）
Anomaly detection from graph data has drawn much attention due to its practical significance in many critical applications including cybersecurity, finance, and social networks. Existing data mining and machine learning methods are either shallow methods that could not effectively capture the complex interdependency of graph data or graph autoencoder methods that could not fully exploit the contextual information as supervision signals for effective anomaly detection. To overcome these challenges, in this paper, we propose a novel method, Self-Supervised Learning for Graph Anomaly Detection (S…
Graph Mixture of Experts and Memory-augmented Routers for Multivariate Time Series Anomaly Detection
University of Science and Technology of China（中国科学技术大学）；Institute of Plasma Physics, Hefei Institutes of Physical Science, Chinese Academy of Sciences（中国科学院合肥物质科学研究院等离子体物理研究所）
Text-Attributed Graph Anomaly Detection via Multi-Scale Cross- and Uni-Modal Contrastive Learning
School of Computer Science and Technology, Xi'an Jiaotong University（西安交通大学计算机科学与技术学院）；Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering, Xi’an Jiaotong University（陕西省大数据知识工程重点实验室）；School of Distance Education, Xi’an Jiaotong University（西安交通大学继续教育学院）；University of Virginia（弗吉尼亚大学）
Monash University（莫纳什大学）；Alibaba Group（阿里巴巴集团）；Nanjing University of Science and Technology（南京理工大学）；The Hong Kong Polytechnic University（香港理工大学）；Chinese Academy of Sciences（中国科学院）；University of Minnesota（明尼苏达大学）
Anomaly detection on attributed networks attracts considerable research interests due to wide applications of attributed networks in modeling a wide range of complex systems. Recently, the deep learning-based anomaly detection methods have shown promising results over shallow approaches, especially on networks with high-dimensional attributes and complex structures. However, existing approaches, which employ graph autoencoder as their backbone, do not fully exploit the rich information of the network, resulting in suboptimal performance. Furthermore, these methods do not directly target anomal…
