LTCI, Télécom Paris, Institut Polytechnique de Paris（巴黎高等电信学院 LTCI 实验室，巴黎综合理工学院）；Department of Operations and Information Systems, University of Graz（格拉茨大学运营与信息系统系）；Airbus AI Research（空客人工智能研究院）
Guillaume Staerman, Eric Adjakossa, Pavlo Mozharovskyi, Vera Hofer, Jayant Sen Gupta, Stephan Clémençon
The increasing automation in many areas of the Industry expressly demands to design efficient machine-learning solutions for the detection of abnormal events. With the ubiquitous deployment of sensors monitoring nearly continuously the health of complex infrastructures, anomaly detection can now rely on measurements sampled at a very high frequency, providing a very rich representation of the phenomenon under surveillance. In order to exploit fully the information thus collected, the observations cannot be treated as multivariate data anymore and a functional analysis approach is required. It is the purpose of this paper to investigate the performance of recent techniques for anomaly detection in the functional setup on real datasets. After an overview of the state-of-the-art and a visual-descriptive study, a variety of anomaly detection methods are compared. While taxonomies of abnormalities (e.g. shape, location) in the functional setup are documented in the literature, assigning a specific type to the identified anomalies appears to be a challenging task. Thus, strengths and weaknesses of the existing approaches are benchmarked in view of these highlighted types in a simulation study. Anomaly detection methods are next evaluated on two datasets, related to the monitoring of helicopters in flight and to the spectrometry of construction materials namely. The benchmark analysis is concluded by recommendation guidance for practitioners.
Optimistik；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Univ. Savoie Mont Blanc（萨瓦大学）；CNRS（法国国家科学研究中心）
In this paper we describe an approach for anomaly detection and its explainability in multivariate functional data. The anomaly detection procedure consists of transforming the series into a vector of features and using an Isolation forest algorithm. The explainable procedure is based on the computation of the SHAP coefficients and on the use of a supervised decision tree. We apply it on simulated data to measure the performance of our method and on real data coming from industry.
Time series anomaly detection has been a perennially important topic in data science, with papers dating back to the 1950s. However, in recent years there has been an explosion of interest in this topic, much of it driven by the success of deep learning in other domains and for other time series tasks. Most of these papers test on one or more of a handful of popular benchmark datasets, created by Yahoo, Numenta, NASA, etc. In this work we make a surprising claim. The majority of the individual exemplars in these datasets suffer from one or more of four flaws. Because of these four flaws, we be…
Standard methods for anomaly detection assume that all features are observed at both learning time and prediction time. Such methods cannot process data containing missing values. This paper studies five strategies for handling missing values in test queries: (a) mean imputation, (b) MAP imputation, (c) reduction (reduced-dimension anomaly detectors via feature bagging), (d) marginalization (for density estimators only), and (e) proportional distribution (for tree-based methods only). Our analysis suggests that MAP imputation and proportional distribution should give better results than mean i…
LLmFPCA-detect: LLM-powered Multivariate Functional PCA for Anomaly Detection in Sparse Longitudinal Texts
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology（乔治亚理工学院H.Milton Stewart工业与系统工程学院）；AT&T Chief Data Office（AT&T首席数据办公室）；Department of Data Sciences and Operations, Marshall School of Business, University of Southern California（南加州大学马歇尔商学院数据科学与运营学院）
Sichuan University（四川大学）；University of Electronic Science and Technology of China（电子科技大学）
Time series anomaly detection strives to uncover potential abnormal behaviors and patterns from temporal data, and has fundamental significance in diverse application scenarios. Constructing an effective detection model usually requires adequate training data stored in a centralized manner, however, this requirement sometimes could not be satisfied in realistic scenarios. As a prevailing approach to address the above problem, federated learning has demonstrated its power to cooperate with the distributed data available while protecting the privacy of data providers. However, it is still unclea…
This article provides a thorough meta-analysis of the anomaly detection problem. To accomplish this we first identify approaches to benchmarking anomaly detection algorithms across the literature and produce a large corpus of anomaly detection benchmarks that vary in their construction across several dimensions we deem important to real-world applications: (a) point difficulty, (b) relative frequency of anomalies, (c) clusteredness of anomalies, and (d) relevance of features. We apply a representative set of anomaly detection algorithms to this corpus, yielding a very large collection of exper…
Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations; examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, not batches, and learn while simultaneously making predictions. There are no benchmarks to adequately test and score the efficacy of real-time anomaly detectors. Here we propose the Numenta Anomaly Benchmark (NAB), which attempts to provide a controlled and repeatable environment o…
Shanghai University of Finance and Economics（上海财经大学）；Carnegie Mellon University（卡内基梅隆大学）
Given a long list of anomaly detection algorithms developed in the last few decades, how do they perform with regard to (i) varying levels of supervision, (ii) different types of anomalies, and (iii) noisy and corrupted data? In this work, we answer these key questions by conducting (to our best knowledge) the most comprehensive anomaly detection benchmark with 30 algorithms on 57 benchmark datasets, named ADBench. Our extensive experiments (98,436 in total) identify meaningful insights into the role of supervision and anomaly types, and unlock future directions for researchers in algorithm se…
Why Ranking Anomaly Detection Algorithms Isn't as Reliable as You May Think
Carnegie Mellon University（卡内基梅隆大学）；TU Dortmund University（多特蒙德工业大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）；iits Consulting GmbH（iits咨询有限公司）；Fraunhofer Institute for Material Flow and Logistics IML（弗劳恩霍夫物流与材料流动研究所IML）
UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detection
University of Bucharest（布加勒斯特大学）；MBZ University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；SecurifAI；Linköping University（林雪平大学）；University of Central Florida（中佛罗里达大学）
Access to high-quality data repositories and benchmarks have been instrumental in advancing the state of the art in many experimental research domains. While advanced analytics tasks over time series data have been gaining lots of attention, lack of such community resources severely limits scientific progress. In this paper, we present Exathlon, the first comprehensive public benchmark for explainable anomaly detection over high-dimensional time series data. Exathlon has been systematically constructed based on real data traces from repeated executions of large-scale stream processing jobs on …
Are Anomaly Scores Telling the Whole Story? A Benchmark for Multilevel Anomaly Detection
National University of Singapore（新加坡国立大学）；Singapore University of Technology and Design（新加坡科技设计大学）；University of Science（科学大学）；Vietnam National University, Ho Chi Minh City（胡志明市越南国家大学）
With the high requirements of automation in the era of Industry 4.0, anomaly detection plays an increasingly important role in higher safety and reliability in the production and manufacturing industry. Recently, autoencoders have been widely used as a backend algorithm for anomaly detection. Different techniques have been developed to improve the anomaly detection performance of autoencoders. Nonetheless, little attention has been paid to the latent representations learned by autoencoders. In this paper, we propose a novel selection-and-weighting-based anomaly detection framework called SWAD.…
European Space Agency Benchmark for Anomaly Detection in Satellite Telemetry
KP Labs；Airbus Defence and Space GmbH（空客防务与航天有限公司）；Opole University of Technology（奥波莱理工大学）；Silesian University of Technology（西里西亚理工大学）；CGI Deutschland B.V. & Co. KG（CGI 德国有限两合公司）；European Space Operations Centre, European Space Agency（欧洲空间局欧洲空间操作中心）；Telespazio Germany GmbH（意大利空间公司德国有限公司）；LSE Space GmbH（LSE Space 有限公司）；Solenix GmbH（Solenix 有限公司）
BenchReAD: A systematic benchmark for retinal anomaly detection
The Center for Smart Health, School of Nursing, the Hong Kong Polytechnic University, Hong Kong, China（香港理工大学护理学院智能健康中心）；School of Biomedical Engineering, Tsinghua University, Beijing, China（清华大学生物医学工程学院）；Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China（中国科学院深圳先进技术研究院医学人工智能研究中心）
Universität Hamburg（汉堡大学）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Rutgers University（罗格斯大学）
The identification of anomalous overdensities in data - group or collective anomaly detection - is a rich problem with a large number of real world applications. However, it has received relatively little attention in the broader ML community, as compared to point anomalies or other types of single instance outliers. One reason for this is the lack of powerful benchmark datasets. In this paper, we first explain how, after the Nobel-prize winning discovery of the Higgs boson, unsupervised group anomaly detection has become a new frontier of fundamental physics (where the motivation is to find n…
School of Computer Science, McGill University（麦吉尔大学计算机科学学院）；Department of Physics, University of Montreal（蒙特利尔大学物理系）；Mila - Quebec AI Institute（魁北克人工智能研究所）
Warsaw University of Technology（华沙理工大学）；TECNALIA（泰克纳利亚）；University of the Basque Country (UPV/EHU)（巴斯克大学(UPV/EHU)）
Unsupervised anomaly discovery in stream data is a research topic with many practical applications. However, in many cases, it is not easy to collect enough training data with labeled anomalies for supervised learning of an anomaly detector in order to deploy it later for identification of real anomalies in streaming data. It is thus important to design anomalies detectors that can correctly detect anomalies without access to labeled training data. Our idea is to adapt the Online evolving Spiking Neural Network (OeSNN) classifier to the anomaly detection task. As a result, we offer an Online e…
TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Models
Computer Network Information Center, CAS（中国科学院计算机网络信息中心）；Nankai University（南开大学）；University of Chinese Academy of Sciences（中国科学院大学）；Jilin University（吉林大学）；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院）；Tsinghua University（清华大学）；ZTE Co., Ltd（中兴通讯股份有限公司）
University of the Witwatersrand（金山大学）；University of Johannesburg（约翰内斯堡大学）
Hybrid methods have been shown to outperform pure statistical and pure deep learning methods at both forecasting tasks, and at quantifying the uncertainty associated with those forecasts (prediction intervals). One example is Multivariate Exponential Smoothing Long Short-Term Memory (MES-LSTM), a hybrid between a multivariate statistical forecasting model and a Recurrent Neural Network variant, Long Short-Term Memory. It has also been shown that a model that ($i$) produces accurate forecasts and ($ii$) is able to quantify the associated predictive uncertainty satisfactorily, can be successfull…
Shanghai AI Laboratory（上海人工智能实验室）；Huawei Cloud & AI（华为云与人工智能）
This paper explores semi-supervised anomaly detection, a more practical setting for anomaly detection where a small additional set of labeled samples are provided. We propose a new KL-divergence based objective function for semi-supervised anomaly detection, and show that two factors: the mutual information between the data and latent representations, and the entropy of latent representations, constitute an integral objective function for anomaly detection. To resolve the contradiction in simultaneously optimizing the two factors, we propose a novel encoder-decoder-encoder structure, with the …
Functional Localization Enforced Deep Anomaly Detection Using Fundus Images
Heinrich Heine University Düsseldorf（杜塞尔多夫海因里希·海涅大学）；Philipps University of Marburg（马尔堡菲利普斯大学）；University Hospital Knappschaftskrankenhaus Bochum（波鸿克纳普schaft医院大学医院）；University of Münster（明斯特大学）
