8 pages, 6 figures. To appear in Proceedings 35th AAAI Conference on Artificial Intelligence (AAAI 21)
发表信息：
Proceedings of the AAAI Conference on Artificial Intelligence, 35(5), 4090-4097 (2021). Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16530
When formulated as an unsupervised learning problem, anomaly detection often requires a model to learn the distribution of normal data. Previous works apply Generative Adversarial Networks (GANs) to anomaly detection tasks and show good performances from these models. Motivated by the observation that GAN ensembles often outperform single GANs in generation tasks, we propose to construct GAN ensembles for anomaly detection. In the proposed method, a group of generators and a group of discriminators are trained together, so every generator gets feedback from multiple discriminators, and vice versa. Compared to a single GAN, a GAN ensemble can better model the distribution of normal data and thus better detect anomalies. Our theoretical analysis of GANs and GAN ensembles explains the role of a GAN discriminator in anomaly detection. In the empirical study, we evaluate ensembles constructed from four types of base models, and the results show that these ensembles clearly outperform single models in a series of tasks of anomaly detection.
Sun Yat-sen University（中山大学）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）
The goal of anomaly detection is to identify anomalous samples from normal ones. In this paper, a small number of anomalies are assumed to be available at the training stage, but they are assumed to be collected only from several anomaly types, leaving the majority of anomaly types not represented in the collected anomaly dataset at all. To effectively leverage this kind of incomplete anomalous knowledge represented by the collected anomalies, we propose to learn a probability distribution that can not only model the normal samples, but also guarantee to assign low density values for the colle…
Anomaly detection is a significant problem faced in several research areas. Detecting and correctly classifying something unseen as anomalous is a challenging problem that has been tackled in many different manners over the years. Generative Adversarial Networks (GANs) and the adversarial training process have been recently employed to face this task yielding remarkable results. In this paper we survey the principal GAN-based anomaly detection methods, highlighting their pros and cons. Our contributions are the empirical validation of the main GAN models for anomaly detection, the increase of …
Airbus AI Research（空客人工智能研究院）；ISAE Supaero（法国高等航空航天学院）
Generative adversarial networks are a class of generative algorithms that have been widely used to produce state-of-the-art samples. In this paper, we investigate GAN to perform anomaly detection on time series dataset. In order to achieve this goal, a bibliography is made focusing on theoretical properties of GAN and GAN used for anomaly detection. A Wasserstein GAN has been chosen to learn the representation of normal data distribution and a stacked encoder with the generator performs the anomaly detection. W-GAN with encoder seems to produce state of the art anomaly detection scores on MNIS…
Bioinformatics Institute, Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局生物信息学研究所）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；Hwa Chong Institution (College Section)（华侨中学（高中部））
Anomaly detection is a classical problem where the aim is to detect anomalous data that do not belong to the normal data distribution. Current state-of-the-art methods for anomaly detection on complex high-dimensional data are based on the generative adversarial network (GAN). However, the traditional GAN loss is not directly aligned with the anomaly detection objective: it encourages the distribution of the generated samples to overlap with the real data and so the resulting discriminator has been found to be ineffective as an anomaly detector. In this paper, we propose simple modifications t…
CentraleSupélec（中央理工-高等电力学院）；Institute for Infocomm Research（资讯通信研究所）；Télécom ParisTech（巴黎高等电信学院）；Carnegie Mellon University（卡内基梅隆大学）；Nanyang Technological University（南洋理工大学）
Generative adversarial networks (GANs) are able to model the complex highdimensional distributions of real-world data, which suggests they could be effective for anomaly detection. However, few works have explored the use of GANs for the anomaly detection task. We leverage recently developed GAN models for anomaly detection, and achieve state-of-the-art performance on image and network intrusion datasets, while being several hundred-fold faster at test time than the only published GAN-based method.
North Carolina State University（北卡罗来纳州立大学）；Intuit Inc.（财捷公司）
Robust anomaly detection is a requirement for monitoring complex modern systems with applications such as cyber-security, fraud prevention, and maintenance. These systems generate multiple correlated time series that are highly seasonal and noisy. This paper presents a novel unsupervised deep learning architecture for multivariate time series anomaly detection, called Robust Seasonal Multivariate Generative Adversarial Network (RSM-GAN). It extends recent advancements in GANs with adoption of convolutional-LSTM layers and an attention mechanism to produce state-of-the-art performance. We condu…
IE–FING, UDELAR（乌拉圭共和国大学工程学院信息工程系）；AIT Austrian Institute of Technology（奥地利技术研究院）
Despite the many attempts and approaches for anomaly detection explored over the years, the automatic detection of rare events in data communication networks remains a complex problem. In this paper we introduce Net-GAN, a novel approach to network anomaly detection in time-series, using recurrent neural networks (RNNs) and generative adversarial networks (GAN). Different from the state of the art, which traditionally focuses on univariate measurements, Net-GAN detects anomalies in multivariate time-series, exploiting temporal dependencies through RNNs. Net-GAN discovers the underlying distrib…
MIM-GAN-based Anomaly Detection for Multivariate Time Series Data
Tibet University（西藏大学）；Jinan University（暨南大学）；Western University（韦仕敦大学）；Tibet Autonomous Region Plateau Complex Environmental Information Transmission and Processing Engineering Research Center（西藏自治区高原复杂环境信息传输与处理工程研究中心）；Advanced Technology Research Center of Tibet University（西藏大学先进技术研究中心）
Machine learning (ML) algorithms are optimized for the distribution represented by the training data. For outlier data, they often deliver predictions with equal confidence, even though these should not be trusted. In order to deploy ML-based digital pathology solutions in clinical practice, effective methods for detecting anomalous data are crucial to avoid incorrect decisions in the outlier scenario. We propose a new unsupervised learning approach for anomaly detection in histopathology data based on generative adversarial networks (GANs). Compared to the existing GAN-based methods that have…
University of Colorado at Boulder（科罗拉多大学博尔德分校）；University of Maryland, College Park（马里兰大学帕克分校）；IUPUI（印第安纳大学与普渡大学印第安纳波利斯联合分校）
Detecting anomalous activity in human mobility data has a number of applications including road hazard sensing, telematic based insurance, and fraud detection in taxi services and ride sharing. In this paper we address two challenges that arise in the study of anomalous human trajectories: 1) a lack of ground truth data on what defines an anomaly and 2) the dependence of existing methods on significant pre-processing and feature engineering. While generative adversarial networks seem like a natural fit for addressing these challenges, we find that existing GAN based anomaly detection algorithm…
Programmable logic controller (PLC) based industrial control systems (ICS) are used to monitor and control critical infrastructure. Integration of communication networks and an Internet of Things approach in ICS has increased ICS vulnerability to cyber-attacks. This work proposes novel unsupervised machine learning ensemble methods for anomaly detection in PLC-based ICS. The work presents two broad approaches to anomaly detection: a weighted voting ensemble approach with a learning algorithm based on coefficient of determination and a stacking-based ensemble approach using isolation forest met…
Constructing an ensemble from a heterogeneous set of unsupervised anomaly detection methods is challenging because the class labels or the ground truth is unknown. Thus, traditional ensemble techniques that use the response variable or the class labels cannot be used to construct an ensemble for unsupervised anomaly detection. We use Item Response Theory (IRT) -- a class of models used in educational psychometrics to assess student and test question characteristics -- to construct an unsupervised anomaly detection ensemble. IRT's latent trait computation lends itself to anomaly detection becau…
As large-scale time-series data can easily be found in real-world applications, multivariate time-series anomaly detection has played an essential role in diverse industries. It enables productivity improvement and maintenance cost reduction by preventing malfunctions and detecting anomalies based on time-series data. However, multivariate time-series anomaly detection is challenging because real-world time-series data exhibit complex temporal dependencies. For this task, it is crucial to learn a rich representation that effectively contains the nonlinear temporal dynamics of normal behavior. …
Multivariate time series anomaly detection is a very common problem in the field of failure prevention. Fast prevention means lower repair costs and losses. The amount of sensors in novel industry systems makes the anomaly detection process quite difficult for humans. Algorithms which automates the process of detecting anomalies are crucial in modern failure-prevention systems. Therefore, many machine and deep learning models have been designed to address this problem. Mostly, they are autoencoder-based architectures with some generative adversarial elements. In this work, a framework is shown…
CentraleSupélec（中央理工-高等电力学院）；École Polytechnique（巴黎综合理工学院）；Télécom ParisTech（巴黎电信学院）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）
Anomaly detection is a significant and hence well-studied problem. However, developing effective anomaly detection methods for complex and high-dimensional data remains a challenge. As Generative Adversarial Networks (GANs) are able to model the complex high-dimensional distributions of real-world data, they offer a promising approach to address this challenge. In this work, we propose an anomaly detection method, Adversarially Learned Anomaly Detection (ALAD) based on bi-directional GANs, that derives adversarially learned features for the anomaly detection task. ALAD then uses reconstruction…
In this study, we investigate the effectiveness of advanced feature engineering and hybrid model architectures for anomaly detection in a multivariate industrial time series, focusing on a steam turbine system. We evaluate the impact of change point-derived statistical features, clustering-based substructure representations, and hybrid learning strategies on detection performance. Despite their theoretical appeal, these complex approaches consistently underperformed compared to a simple Random Forest + XGBoost ensemble trained on segmented data. The ensemble achieved an AUC-ROC of 0.976, F1-sc…
Anomaly detection incurs certain sampling and sensing costs and therefore it is of great importance to strike a balance between the detection accuracy and these costs. In this work, we study anomaly detection by considering the detection of threshold crossings in a stochastic time series without the knowledge of its statistics. To reduce the sampling cost in this detection process, we propose the use of hierarchical generative adversarial networks (GANs) to perform nonuniform sampling. In order to improve the detection accuracy and reduce the delay in detection, we introduce a buffer zone in t…
Quantum enhanced ensemble GANs for anomaly detection in continuous biomanufacturing
Department of Chemical and Biochemical Engineering, Technical University of Denmark（化学与生物化学工程系，丹麦技术大学）；ORCA Computing（ORCA计算公司）；Novo Nordisk（诺和制药）；Novo Nordisk Foundation Centre for Biosustainability, Technical University of Denmark（诺和制药可持续发展基金会，丹麦技术大学）
Nowadays, many classification algorithms have been applied to various industries to help them work out their problems met in real-life scenarios. However, in many binary classification tasks, samples in the minority class only make up a small part of all instances, which leads to the datasets we get usually suffer from high imbalance ratio. Existing models sometimes treat minority classes as noise or ignore them as outliers encountering data skewing. In order to solve this problem, we propose a bagging ensemble learning framework $ASE$ (Anomaly Scoring Based Ensemble Learning). This framework …
Anomaly detection is the process of finding data points that deviate from a baseline. In a real-life setting, anomalies are usually unknown or extremely rare. Moreover, the detection must be accomplished in a timely manner or the risk of corrupting the system might grow exponentially. In this work, we propose a two level framework for detecting anomalies in sequences of discrete elements. First, we assess whether we can obtain enough information from the statistics collected from the discriminator's layers to discriminate between out of distribution and in distribution samples. We then build a…
ST Electronics - SUTD Cyber Security Laboratory（ST电子—新加坡科技设计大学网络安全实验室）；Singapore University of Technology and Design（新加坡科技设计大学）
Recently, the introduction of the generative adversarial network (GAN) and its variants has enabled the generation of realistic synthetic samples, which has been used for enlarging training sets. Previous work primarily focused on data augmentation for semi-supervised and supervised tasks. In this paper, we instead focus on unsupervised anomaly detection and propose a novel generative data augmentation framework optimized for this task. In particular, we propose to oversample infrequent normal samples - normal samples that occur with small probability, e.g., rare normal events. We show that th…
Enhancing Network Anomaly Detection with Quantum GANs and Successive Data Injection for Multivariate Time Series
Department of Computer Engineering（计算机工程系）；Polytechnique Montreal（蒙特利尔理工学院）；Department of Compute Engineering（计算机工程系）；Department of Computer Science（计算机科学系）；Université de Sherbrooke（谢布鲁克大学）
Information Science and Technology Institute (ISTI) of CNR（意大利国家研究委员会信息科学与技术研究所（ISTI））
In this work, we propose CBiGAN -- a novel method for anomaly detection in images, where a consistency constraint is introduced as a regularization term in both the encoder and decoder of a BiGAN. Our model exhibits fairly good modeling power and reconstruction consistency capability. We evaluate the proposed method on MVTec AD -- a real-world benchmark for unsupervised anomaly detection on high-resolution images -- and compare against standard baselines and state-of-the-art approaches. Experiments show that the proposed method improves the performance of BiGAN formulations by a large margin a…
Center for Applied Intelligent Systems Research (CAISR)（应用智能系统研究中心）；Halmstad University（哈尔姆斯塔德大学）
In contextual anomaly detection, an object is only considered anomalous within a specific context. Most existing methods for CAD use a single context based on a set of user-specified contextual features. However, identifying the right context can be very challenging in practice, especially in datasets, with a large number of attributes. Furthermore, in real-world systems, there might be multiple anomalies that occur in different contexts and, therefore, require a combination of several "useful" contexts to unveil them. In this work, we leverage active learning and ensembles to effectively dete…
We proposed a multivariate time series anomaly detection frame-work Ymir, which leverages ensemble learning and supervisedlearning technology to efficiently learn and adapt to anomaliesin real-world system applications. Ymir integrates several currentlywidely used unsupervised anomaly detection models through anensemble learning method, and thus can provide robust frontalanomaly detection results in unsupervised scenarios. In a super-vised setting, domain experts and system users discuss and providelabels (anomalous or not) for the training data, which reflects theiranomaly detection criteria …
Despite inherent ill-definition, anomaly detection is a research endeavor of great interest within machine learning and visual scene understanding alike. Most commonly, anomaly detection is considered as the detection of outliers within a given data distribution based on some measure of normality. The most significant challenge in real-world anomaly detection problems is that available data is highly imbalanced towards normality (i.e. non-anomalous) and contains a most a subset of all possible anomalous samples - hence limiting the use of well-established supervised learning methods. By contra…
School of Information and Software Engineering, University of Electronic Science and Technology of China（电子科技大学信息与软件工程学院）
KPI anomaly detection is one important function of network management system. Traditional methods either require prior knowledge or manually set thresholds. To overcome these shortcomings, we propose the Ensemble2 framework, which applies ensemble learning to improve exogenous capabilities. Meanwhile, automatically adjusts thresholds based on extreme value theory. The model is tested on production datasets to verify its effectiveness. We further optimize the model using online learning, and finally running at a speed of ~10 pts/s on an Intel i5 platform.
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究所）；PO–AO “Minimaks-94”（PO–AO“Minimaks-94”）
The main aim of this work is to develop and implement an automatic anomaly detection algorithm for meteorological time-series. To achieve this goal we develop an approach to constructing an ensemble of anomaly detectors in combination with adaptive threshold selection based on artificially generated anomalies. We demonstrate the efficiency of the proposed method by integrating the corresponding implementation into ``Minimax-94'' road weather information system.
Forecasting Anomaly Precursors via Uncertainty-Aware Time-Series Ensembles
Department of Industrial & Management Engineering, Korea University（韩国大学工业与管理工程系）；Department of Industrial Engineering, Seoul National University（首尔国立大学工业工程系）；LG CNS
Isolation-based Spherical Ensemble Representations for Tabular Anomaly Detection
Tsinghua Shenzhen International Graduate School, Tsinghua University, China（清华大学深圳国际研究生院，清华大学，中国）；School of Computing and Information Technology, Great Bay University, China（计算机与信息科技学院，Great Bay大学，中国）；School of Compuer Science, Nanjing University, China（计算机科学学院，南京大学，中国）；Computer department, Dongguan university of technology, China（计算机系，东莞科技学院，中国）；College of Computer Science and Technology, China University of Petroleum, China（计算机科学与技术学院，中国石油大学，中国）；School of IT, Deakin University, Australia（信息技术学院，德金大学，澳大利亚）
In critical applications of anomaly detection including computer security and fraud prevention, the anomaly detector must be configurable by the analyst to minimize the effort on false positives. One important way to configure the anomaly detector is by providing true labels for a few instances. We study the problem of label-efficient active learning to automatically tune anomaly detection ensembles and make four main contributions. First, we present an important insight into how anomaly detector ensembles are naturally suited for active learning. This insight allows us to relate the greedy qu…
Columbia University（哥伦比亚大学）；Facebook Data Engineering（Facebook数据工程）
Methods for unsupervised anomaly detection suffer from the fact that the data is unlabeled, making it difficult to assess the optimality of detection algorithms. Ensemble learning has shown exceptional results in classification and clustering problems, but has not seen as much research in the context of outlier detection. Existing methods focus on combining output scores of individual detectors, but this leads to outputs that are not easily interpretable. In this paper, we introduce a theoretical foundation for combining individual detectors with Bayesian classifier combination. Not only are p…
LPIXEL Inc.（LPIXEL公司）；University of Cambridge（剑桥大学）；National Institute of Informatics（国立信息学研究所）；National Center for Global Health and Medicine（国立国际医疗研究中心）；Juntendo University（顺天堂大学）；ELTE Eötvös Loránd University（厄特沃什·罗兰大学）；The University of Tokyo（东京大学）
Unsupervised learning can discover various unseen abnormalities, relying on large-scale unannotated medical images of healthy subjects. Towards this, unsupervised methods reconstruct a 2D/3D single medical image to detect outliers either in the learned feature space or from high reconstruction loss. However, without considering continuity between multiple adjacent slices, they cannot directly discriminate diseases composed of the accumulation of subtle anatomical anomalies, such as Alzheimer's Disease (AD). Moreover, no study has shown how unsupervised anomaly detection is associated with eith…
Identifying anomalies in multi-dimensional datasets is an important task in many real-world applications. A special case arises when anomalies are occluded in a small set of attributes, typically referred to as a subspace, and not necessarily over the entire data space. In this paper, we propose a new subspace analysis approach named Agglomerative Attribute Grouping (AAG) that aims to address this challenge by searching for subspaces that are comprised of highly correlative attributes. Such correlations among attributes represent a systematic interaction among the attributes that can better re…
Eindhoven University of Technology（埃因霍温理工大学）；Royal Philips B. V.（皇家飞利浦有限公司）
Many important data analysis applications present with severely imbalanced datasets with respect to the target variable. A typical example is medical image analysis, where positive samples are scarce, while performance is commonly estimated against the correct detection of these positive examples. We approach this challenge by formulating the problem as anomaly detection with generative models. We train a generative model without supervision on the `negative' (common) datapoints and use this model to estimate the likelihood of unseen data. A successful model allows us to detect the `positive' …
McGill University（麦吉尔大学）；Hydro-Québec’s research institute（魁北克水电研究院）
Hacking and false data injection from adversaries can threaten power grids' everyday operations and cause significant economic loss. Anomaly detection in power grids aims to detect and discriminate anomalies caused by cyber attacks against the power system, which is essential for keeping power grids working correctly and efficiently. Different methods have been applied for anomaly detection, such as statistical methods and machine learning-based methods. Usually, machine learning-based methods need to model the normal data distribution. In this work, we propose a novel anomaly detection method…
In this paper, we propose POTATOES (Partitioning OverfiTting AuTOencoder EnSemble), a new method for unsupervised outlier detection (UOD). More precisely, given any autoencoder for UOD, this technique can be used to improve its accuracy while at the same time removing the burden of tuning its regularization. The idea is to not regularize at all, but to rather randomly partition the data into sufficiently many equally sized parts, overfit each part with its own autoencoder, and to use the maximum over all autoencoder reconstruction errors as the anomaly score. We apply our model to various real…
