The problem of detecting data anomaly is considered. Under the null hypothesis that models anomaly-free data, measurements are assumed to be from an unknown distribution with some authenticated historical samples. Under the composite alternative hypothesis, measurements are from an unknown distribution positive distance away from the distribution under the null hypothesis. No training data are available for the distribution of anomaly data. A semi-supervised deep learning technique based on an inverse generative adversary network is proposed.
A method for unsupervised contextual anomaly detection is proposed using a cross-linked pair of Variational Auto-Encoders for assigning a normality score to an observation. The method enables a distinct separation of contextual from behavioral attributes and is robust to the presence of anomalous or novel contextual attributes. The method can be trained with data sets that contain anomalies without any special pre-processing.
Many deep models have been recently proposed for anomaly detection. This paper presents comparison of selected generative deep models and classical anomaly detection methods on an extensive number of non--image benchmark datasets. We provide statistical comparison of the selected models, in many configurations, architectures and hyperparamaters. We arrive to conclusion that performance of the generative models is determined by the process of selection of their hyperparameters. Specifically, performance of the deep generative models deteriorates with decreasing amount of anomalous samples used …
Eindhoven University of Technology（埃因霍温理工大学）；Royal Philips B. V.（皇家飞利浦有限公司）
Many important data analysis applications present with severely imbalanced datasets with respect to the target variable. A typical example is medical image analysis, where positive samples are scarce, while performance is commonly estimated against the correct detection of these positive examples. We approach this challenge by formulating the problem as anomaly detection with generative models. We train a generative model without supervision on the `negative' (common) datapoints and use this model to estimate the likelihood of unseen data. A successful model allows us to detect the `positive' …
Machine learning models encounter Out-of-Distribution (OoD) errors when the data seen at test time are generated from a different stochastic generator than the one used to generate the training data. One proposal to scale OoD detection to high-dimensional data is to learn a tractable likelihood approximation of the training distribution, and use it to reject unlikely inputs. However, likelihood models on natural data are themselves susceptible to OoD errors, and even assign large likelihoods to samples from other datasets. To mitigate this problem, we propose Generative Ensembles, which robust…
ST Electronics - SUTD Cyber Security Laboratory（ST电子—新加坡科技设计大学网络安全实验室）；Singapore University of Technology and Design（新加坡科技设计大学）
Recently, the introduction of the generative adversarial network (GAN) and its variants has enabled the generation of realistic synthetic samples, which has been used for enlarging training sets. Previous work primarily focused on data augmentation for semi-supervised and supervised tasks. In this paper, we instead focus on unsupervised anomaly detection and propose a novel generative data augmentation framework optimized for this task. In particular, we propose to oversample infrequent normal samples - normal samples that occur with small probability, e.g., rare normal events. We show that th…
Accurate and automated detection of anomalous samples in a natural image dataset can be accomplished with a probabilistic model for end-to-end modeling of images. Such images have heterogeneous complexity, however, and a probabilistic model overlooks simply shaped objects with small anomalies. This is because the probabilistic model assigns undesirably lower likelihoods to complexly shaped objects that are nevertheless consistent with set standards. To overcome this difficulty, we propose an unregularized score for deep generative models (DGMs), which are generative models leveraging deep neur…
Traffic congestion anomaly detection is of paramount importance in intelligent traffic systems. The goals of transportation agencies are two-fold: to monitor the general traffic conditions in the area of interest and to locate road segments under abnormal congestion states. Modeling congestion patterns can achieve these goals for citywide roadways, which amounts to learning the distribution of multivariate time series (MTS). However, existing works are either not scalable or unable to capture the spatial-temporal information in MTS simultaneously. To this end, we propose a principled and compr…
University of Maryland College Park（马里兰大学帕克分校）；IBM Research（IBM研究院）
Detecting out of distribution (OOD) samples is of paramount importance in all Machine Learning applications. Deep generative modeling has emerged as a dominant paradigm to model complex data distributions without labels. However, prior work has shown that generative models tend to assign higher likelihoods to OOD samples compared to the data distribution on which they were trained. First, we propose Adversarial Mirrored Autoencoder (AMA), a variant of Adversarial Autoencoder, which uses a mirrored Wasserstein loss in the discriminator to enforce better semantic-level reconstruction. We also pr…
Airbus AI Research（空客人工智能研究院）；ISAE Supaero（法国高等航空航天学院）
Generative adversarial networks are a class of generative algorithms that have been widely used to produce state-of-the-art samples. In this paper, we investigate GAN to perform anomaly detection on time series dataset. In order to achieve this goal, a bibliography is made focusing on theoretical properties of GAN and GAN used for anomaly detection. A Wasserstein GAN has been chosen to learn the representation of normal data distribution and a stacked encoder with the generator performs the anomaly detection. W-GAN with encoder seems to produce state of the art anomaly detection scores on MNIS…
IE–FING, UDELAR（乌拉圭共和国大学工程学院信息工程系）；AIT Austrian Institute of Technology（奥地利技术研究院）
Despite the many attempts and approaches for anomaly detection explored over the years, the automatic detection of rare events in data communication networks remains a complex problem. In this paper we introduce Net-GAN, a novel approach to network anomaly detection in time-series, using recurrent neural networks (RNNs) and generative adversarial networks (GAN). Different from the state of the art, which traditionally focuses on univariate measurements, Net-GAN detects anomalies in multivariate time-series, exploiting temporal dependencies through RNNs. Net-GAN discovers the underlying distrib…
Network anomaly detection is a very relevant research area nowadays, especially due to its multiple applications in the field of network security. The boost of new models based on variational autoencoders and generative adversarial networks has motivated a reevaluation of traditional techniques for anomaly detection. It is, however, essential to be able to understand these new models from the perspective of the experience attained from years of evaluating network security data for anomaly detection. In this paper, we revisit anomaly detection techniques based on PCA from a probabilistic genera…
CausalTAD: Causal Implicit Generative Model for Debiased Online Trajectory Anomaly Detection
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；DiDi Global Inc.（滴滴全球公司）
Tel Aviv University（特拉维夫大学）；Facebook AI Research（Facebook人工智能研究院）
Anomaly detection, the task of identifying unusual samples in data, often relies on a large set of training samples. In this work, we consider the setting of few-shot anomaly detection in images, where only a few images are given at training. We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. We further enhance the representation of our model by using image transformations and optimize scale-specific patch-discriminators to distinguish between real and fake patches of the image, as well as between different transformations applied…
University of Colorado at Boulder（科罗拉多大学博尔德分校）；University of Maryland, College Park（马里兰大学帕克分校）；IUPUI（印第安纳大学与普渡大学印第安纳波利斯联合分校）
Detecting anomalous activity in human mobility data has a number of applications including road hazard sensing, telematic based insurance, and fraud detection in taxi services and ride sharing. In this paper we address two challenges that arise in the study of anomalous human trajectories: 1) a lack of ground truth data on what defines an anomaly and 2) the dependence of existing methods on significant pre-processing and feature engineering. While generative adversarial networks seem like a natural fit for addressing these challenges, we find that existing GAN based anomaly detection algorithm…
AWS AI Labs（AWS人工智能实验室）；CMU（卡内基梅隆大学）；Amazon Research（亚马逊研究院）；UCLA（加州大学洛杉矶分校）
Multivariate time series anomaly detection has become an active area of research in recent years, with Deep Learning models outperforming previous approaches on benchmark datasets. Among reconstruction-based models, most previous work has focused on Variational Autoencoders and Generative Adversarial Networks. This work presents DGHL, a new family of generative models for time series anomaly detection, trained by maximizing the observed likelihood by posterior sampling and alternating back-propagation. A top-down Convolution Network maps a novel hierarchical latent space to time series windows…
In recent years, specific evaluation metrics for time series anomaly detection algorithms have been developed to handle the limitations of the classical precision and recall. However, such metrics are heuristically built as an aggregate of multiple desirable aspects, introduce parameters and wipe out the interpretability of the output. In this article, we first highlight the limitations of the classical precision/recall, as well as the main issues of the recent event-based metrics -- for instance, we show that an adversary algorithm can reach high precision and recall on almost any dataset und…
NASA Ames Research Center（美国国家航空航天局艾姆斯研究中心）；HP SCDS（惠普SCDS）；Universities Space Research Association（大学空间研究协会）；Harvard University（哈佛大学）
Deep generative learning cannot only be used for generating new data with statistical characteristics derived from input data but also for anomaly detection, by separating nominal and anomalous instances based on their reconstruction quality. In this paper, we explore the performance of three unsupervised deep generative models -- variational autoencoders (VAEs) with Gaussian, Bernoulli, and Boltzmann priors -- in detecting anomalies in flight-operations data of commercial flights consisting of multivariate time series. We devised two VAE models with discrete latent variables (DVAEs), one with…
Technical University of Munich（慕尼黑工业大学）；Helmholtz AI（亥姆霍兹人工智能）；Helmholtz Center Munich（亥姆霍兹慕尼黑中心）；Klinikum Rechts der Isar（伊萨尔河右岸医院）；Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）
Early and accurate disease detection is crucial for patient management and successful treatment outcomes. However, the automatic identification of anomalies in medical images can be challenging. Conventional methods rely on large labeled datasets which are difficult to obtain. To overcome these limitations, we introduce a novel unsupervised approach, called PHANES (Pseudo Healthy generative networks for ANomaly Segmentation). Our method has the capability of reversing anomalies, i.e., preserving healthy tissue and replacing anomalous regions with pseudo-healthy (PH) reconstructions. Unlike rec…
VRVis Zentrum für Virtual Reality und Visualisierung Forschungs-GmbH（VRVis虚拟现实与可视化研究中心有限责任公司）；Aalto University（阿尔托大学）
Unsupervised anomaly detection models which are trained solely by healthy data, have gained importance in the recent years, as the annotation of medical data is a tedious task. Autoencoders and generative adversarial networks are the standard anomaly detection methods that are utilized to learn the data distribution. However, they fall short when it comes to inference and evaluation of the likelihood of test samples. We propose a novel combination of generative models and a probabilistic graphical model. After encoding image samples by autoencoders, the distribution of data is modeled by Rando…
Anomaly detection in time series has been widely researched and has important practical applications. In recent years, anomaly detection algorithms are mostly based on deep-learning generative models and use the reconstruction error to detect anomalies. They try to capture the distribution of normal data by reconstructing normal data in the training phase, then calculate the reconstruction error of test data to do anomaly detection. However, most of them only use the normal data in the training phase and can not ensure the reconstruction process of anomaly data. So, anomaly data can also be we…
School of Computer Science and Technology, Harbin University of Science and Technology（哈尔滨理工大学计算机科学与技术学院）；Fujian Provincial Key Laboratory of Information Processing and Intelligent Control, Minjiang University（闽江学院福建省信息处理与智能控制重点实验室）
Unsupervised anomaly detection aims to identify anomalous samples from highly complex and unstructured data, which is pervasive in both fundamental research and industrial applications. However, most existing methods neglect the complex correlation among data samples, which is important for capturing normal patterns from which the abnormal ones deviate. In this paper, we propose a method of Correlation aware unsupervised Anomaly detection via Deep Gaussian Mixture Model (CADGMM), which captures the complex correlation among data points for high-quality low-dimensional representation learning. …
An Attention-Based Deep Generative Model for Anomaly Detection in Industrial Control Systems
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Department of Computer Science, Universidad de las Fuerzas Armadas ESPE（厄瓜多尔武装部队大学计算机科学系）
Zhejiang University（浙江大学）；Ningbo Research Institute, Zhejiang University（浙江大学宁波研究院）
Anomaly detection (AD) has attracted considerable attention in both academia and industry. Due to the lack of anomalous data in many practical cases, AD is usually solved by first modeling the normal data pattern and then determining if data fit this model. Generative models (GMs) seem a natural tool to achieve this purpose, which learn the normal data distribution and estimate it using a probability density function (PDF). However, some works have observed the ideal performance of such GM-based AD methods. In this paper, we propose a new perspective on the ideal performance of GM-based AD met…
Why the Counterintuitive Phenomenon of Likelihood Rarely Appears in Tabular Anomaly Detection with Deep Generative Models?
Department of Industrial Engineering, University of Yonsei, Republic of Korea, Seoul（延世大学工业工程系）；H. Milton Stewart School of Industrial and Systems Engineering (ISyE), Georgia Institute of Technology, USA, GA（佐治亚理工学院H.米尔顿·斯图尔特工业与系统工程学院）
The University of Tokyo Hospital（东京大学医院）；The University of Tokyo（东京大学）；RIKEN（理化学研究所）
Oversight in medical images is a crucial problem, and timely reporting of medical images is desired. Therefore, an all-purpose anomaly detection method that can detect virtually all types of lesions/diseases in a given image is strongly desired. However, few commercially available and versatile anomaly detection methods for medical images have been provided so far. Recently, anomaly detection methods built upon deep learning methods have been rapidly growing in popularity, and these methods seem to provide reasonable solutions to the problem. However, the workload to label the images necessary…
School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）；Squirrel AI（松鼠AI）；Institute for Infocomm Research, A*STAR, Singapore（新加坡科技研究局信息通信研究所）；A*STAR Centre for Frontier AI Research, Singapore（新加坡科技研究局前沿人工智能研究中心）
Massachusetts Institute of Technology（麻省理工学院）；NSF AI Institute for Artificial Intelligence and Fundamental Interactions（美国国家科学基金会人工智能与基本相互作用人工智能研究所）；University of Regina（里贾纳大学）
Anomaly Detection is becoming increasingly popular within the experimental physics community. At experiments such as the Large Hadron Collider, anomaly detection is at the forefront of finding new physics beyond the Standard Model. This paper details the implementation of a novel Machine Learning architecture, called Flux+Mutability, which combines cutting-edge conditional generative models with clustering algorithms. In the `flux' stage we learn the distribution of a reference class. The `mutability' stage at inference addresses if data significantly deviates from the reference class. We demo…
Dimensionality reduction is a crucial first step for many unsupervised learning tasks including anomaly detection and clustering. Autoencoder is a popular mechanism to accomplish dimensionality reduction. In order to make dimensionality reduction effective for high-dimensional data embedding nonlinear low-dimensional manifold, it is understood that some sort of geodesic distance metric should be used to discriminate the data samples. Inspired by the success of geodesic distance approximators such as ISOMAP, we propose to use a minimum spanning tree (MST), a graph-based algorithm, to approximat…
Anomaly detection is a significant problem faced in several research areas. Detecting and correctly classifying something unseen as anomalous is a challenging problem that has been tackled in many different manners over the years. Generative Adversarial Networks (GANs) and the adversarial training process have been recently employed to face this task yielding remarkable results. In this paper we survey the principal GAN-based anomaly detection methods, highlighting their pros and cons. Our contributions are the empirical validation of the main GAN models for anomaly detection, the increase of …
CentraleSupélec（中央理工-高等电力学院）；Institute for Infocomm Research（资讯通信研究所）；Télécom ParisTech（巴黎高等电信学院）；Carnegie Mellon University（卡内基梅隆大学）；Nanyang Technological University（南洋理工大学）
Generative adversarial networks (GANs) are able to model the complex highdimensional distributions of real-world data, which suggests they could be effective for anomaly detection. However, few works have explored the use of GANs for the anomaly detection task. We leverage recently developed GAN models for anomaly detection, and achieve state-of-the-art performance on image and network intrusion datasets, while being several hundred-fold faster at test time than the only published GAN-based method.
Safeguarding Generative AI Applications in Preclinical Imaging through Hybrid Anomaly Detection
Institute of Informatics, University of Warsaw Alethia XAI Sp. z o.o. Warsaw Poland；School of Law, Seoul National University Shin \& Kim LLC, Seoul Republic of Korea；BIOEMTECH, Athens Medical Informatics Laboratory, School of Medicine, University of Thessaly, Larissa Greece；Institute of Informatics, University of Warsaw；Alethia XAI Sp. z o.o.；School of Law, Seoul National University；Medical Informatics Laboratory, School of Medicine, University of Thessaly, Larissa
University of Oslo（奥斯陆大学）；SimulaMet；OsloMet（奥斯陆城市大学）；UiT（特罗姆瑟大学）
The interest for video anomaly detection systems has gained traction for the past few years. The current approaches use deep learning to perform anomaly detection in videos, but this approach has multiple problems. For starters, deep learning in general has issues with noise, concept drift, explainability, and training data volumes. Additionally, anomaly detection in itself is a complex task and faces challenges such as unknowness, heterogeneity, and class imbalance. Anomaly detection using deep learning is therefore mainly constrained to generative models such as generative adversarial networ…
Deep generative models are challenging the classical methods in the field of anomaly detection nowadays. Every new method provides evidence of outperforming its predecessors, often with contradictory results. The objective of this comparison is twofold: to compare anomaly detection methods of various paradigms with focus on deep generative models, and identification of sources of variability that can yield different results. The methods were compared on popular tabular and image datasets. We identified the main sources of variability to be experimental conditions: i) the type data set (tabular…
Generative models are widely used for unsupervised learning with various applications, including data compression and signal restoration. Training methods for such systems focus on the generality of the network given limited amount of training data. A less researched type of techniques concerns generation of only a single type of input. This is useful for applications such as constraint handling, noise reduction and anomaly detection. In this paper we present a technique to limit the generative capability of the network using negative learning. The proposed method searches the solution in the …
