A Transfer Learning Framework for Anomaly Detection Using Model of Normality
Western University（韦仕敦大学）；National Research Council Canada（加拿大国家研究委员会）
Sulaiman Aburakhia, Tareq Tayeh, Ryan Myers, Abdallah Shami
备注：
7 pages, 4 figures, 2 tables, conference: The 11th Annual IEEE Information Technology, Electronics and Mobile Communication Conference "IEEE IEMCON", Vancouver, Canada, November 2020. IEEE IEMCON'20' best paper award in the category of Industrial Automation and Control Systems Technology
Convolutional Neural Network (CNN) techniques have proven to be very useful in image-based anomaly detection applications. CNN can be used as deep features extractor where other anomaly detection techniques are applied on these features. For this scenario, using transfer learning is common since pretrained models provide deep feature representations that are useful for anomaly detection tasks. Consequentially, anomaly can be detected by applying similarly measure between extracted features and a defined model of normality. A key factor in such approaches is the decision threshold used for detecting anomaly. While most of the proposed methods focus on the approach itself, slight attention has been paid to address decision threshold settings. In this paper, we tackle this problem and propose a welldefined method to set the working-point decision threshold that improves detection accuracy. We introduce a transfer learning framework for anomaly detection based on similarity measure with a Model of Normality (MoN) and show that with the proposed threshold settings, a significant performance improvement can be achieved. Moreover, the framework has low complexity with relaxed computational requirements.
Image reconstruction-based anomaly detection has recently been in the spotlight because of the difficulty of constructing anomaly datasets. These approaches work by learning to model normal features without seeing abnormal samples during training and then discriminating anomalies at test time based on the reconstructive errors. However, these models have limitations in reconstructing the abnormal samples due to their indiscriminate conveyance of features. Moreover, these approaches are not explicitly optimized for distinguishable anomalies. To address these problems, we propose a two-stream de…
Pune Institute of Computer Technology（浦那计算机技术学院）；New York University（纽约大学）
The authors have introduced a novel method for unsupervised anomaly detection that utilises a newly introduced Memory Module in their paper. We validate the authors claim that this helps improve performance by helping the network learn prototypical patterns, and uses the learnt memory to reduce the representation capacity of Convolutional Neural Networks. Further, we validate the efficacy of two losses introduced by the authors, Separateness Loss and Compactness Loss presented to increase the discriminative power of the memory items and the deeply learned features. We test the efficacy with th…
Fudan University（复旦大学）；Henan University of Science and Technology（河南科技大学）；University of Shanghai for Science and Technology（上海理工大学）；Shanghai University of Electric Power（上海电力大学）
Video anomaly detection (VAD) has been intensively studied for years because of its potential applications in intelligent video systems. Existing unsupervised VAD methods tend to learn normality from training sets consisting of only normal videos and regard instances deviating from such normality as anomalies. However, they often consider only local or global normality in the temporal dimension. Some of them focus on learning local spatiotemporal representations from consecutive frames to enhance the representation for normal events. But powerful representation allows these methods to represen…
Fudan University（复旦大学）；China University of Petroleum（中国石油大学）
Video anomaly detection is a challenging task because of diverse abnormal events. To this task, methods based on reconstruction and prediction are wildly used in recent works, which are built on the assumption that learning on normal data, anomalies cannot be reconstructed or predicated as good as normal patterns, namely the anomaly result with more errors. In this paper, we propose to discriminate anomalies from normal ones by the duality of normality-granted optical flow, which is conducive to predict normal frames but adverse to abnormal frames. The normality-granted optical flow is predict…
Existing anomaly detection paradigms overwhelmingly focus on training detection models using exclusively normal data or unlabeled data (mostly normal samples). One notorious issue with these approaches is that they are weak in discriminating anomalies from normal samples due to the lack of the knowledge about the anomalies. Here, we study the problem of few-shot anomaly detection, in which we aim at using a few labeled anomaly examples to train sample-efficient discriminative detection models. To address this problem, we introduce a novel weakly-supervised anomaly detection framework to train …
Video anomaly detection is a challenging task in the computer vision community. Most single task-based methods do not consider the independence of unique spatial and temporal patterns, while two-stream structures lack the exploration of the correlations. In this paper, we propose spatial-temporal memories augmented two-stream auto-encoder framework, which learns the appearance normality and motion normality independently and explores the correlations via adversarial learning. Specifically, we first design two proxy tasks to train the two-stream structure to extract appearance and motion featur…
We address the problem of anomaly detection, that is, detecting anomalous events in a video sequence. Anomaly detection methods based on convolutional neural networks (CNNs) typically leverage proxy tasks, such as reconstructing input video frames, to learn models describing normality without seeing anomalous samples at training time, and quantify the extent of abnormalities using the reconstruction error at test time. The main drawbacks of these approaches are that they do not consider the diversity of normal patterns explicitly, and the powerful representation capacity of CNNs allows to reco…
Despite inherent ill-definition, anomaly detection is a research endeavor of great interest within machine learning and visual scene understanding alike. Most commonly, anomaly detection is considered as the detection of outliers within a given data distribution based on some measure of normality. The most significant challenge in real-world anomaly detection problems is that available data is highly imbalanced towards normality (i.e. non-anomalous) and contains a most a subset of all possible anomalous samples - hence limiting the use of well-established supervised learning methods. By contra…
Recent anomaly detection algorithms have shown powerful performance by adopting frame predicting autoencoders. However, these methods face two challenging circumstances. First, they are likely to be trained to be excessively powerful, generating even abnormal frames well, which leads to failure in detecting anomalies. Second, they are distracted by the large number of objects captured in both foreground and background. To solve these problems, we propose a novel superpixel-based video data transformation technique named Random Superpixel Erasing on Moving Objects (RandomSEMO) and Moving Object…
The University of Adelaide（阿德莱德大学）；Deakin University（迪肯大学）；University of Western Australia（西澳大学）
Deep autoencoder has been extensively used for anomaly detection. Training on the normal data, the autoencoder is expected to produce higher reconstruction error for the abnormal inputs than the normal ones, which is adopted as a criterion for identifying anomalies. However, this assumption does not always hold in practice. It has been observed that sometimes the autoencoder "generalizes" so well that it can also reconstruct anomalies well, leading to the miss detection of anomalies. To mitigate this drawback for autoencoder based anomaly detector, we propose to augment the autoencoder with a …
STNMamba: Mamba-based Spatial-Temporal Normality Learning for Video Anomaly Detection
Academy for Engineering and Technology, Fudan University（复旦大学工程技术研究院）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）
Shandong Key Laboratory of Ubiquitous Intelligent Computing, University of Jinan（山东 ubiquitous 智能计算重点实验室，济南大学）；Department of Mechanical Engineering, Shandong University（山东大学机械工程学院）
针对逻辑异常检测难题，提出正态性先验引导的多语义融合网络，利用预训练视觉-语言模型和语义码本注入正常语义，引导异常重建趋近正常，在MVTec LOCO AD上达到SOTA。
Video anomaly detection aims to find the events in a video that do not conform to the expected behavior. The prevalent methods mainly detect anomalies by snippet reconstruction or future frame prediction error. However, the error is highly dependent on the local context of the current snippet and lacks the understanding of normality. To address this issue, we propose to detect anomalous events not only by the local context, but also according to the consistency between the testing event and the knowledge about normality from the training data. Concretely, we propose a novel two-stream framewor…
Anomaly detection consists in identifying, within a dataset, those samples that significantly differ from the majority of the data, representing the normal class. It has many practical applications, e.g. ranging from defective product detection in industrial systems to medical imaging. This paper focuses on image anomaly detection using a deep neural network with multiple pyramid levels to analyze the image features at different scales. We propose a network based on encoding-decoding scheme, using a standard convolutional autoencoders, trained on normal data only in order to build a model of n…
Institute of Imaging & Computer Vision（成像与计算机视觉研究所）；RWTH Aachen University（亚琛工业大学）
Anomaly Detection (AD) in images is a fundamental computer vision problem and refers to identifying images and image substructures that deviate significantly from the norm. Popular AD algorithms commonly try to learn a model of normality from scratch using task specific datasets, but are limited to semi-supervised approaches employing mostly normal data due to the inaccessibility of anomalies on a large scale combined with the ambiguous nature of anomaly appearance. We follow an alternative approach and demonstrate that deep feature representations learned by discriminative models on large nat…
Defence Science and Technology Organization (DSTO)（国防科学技术组织）
Automated detection of abnormalities in data has been studied in research area in recent years because of its diverse applications in practice including video surveillance, industrial damage detection and network intrusion detection. However, building an effective anomaly detection system is a non-trivial task since it requires to tackle challenging issues of the shortage of annotated data, inability of defining anomaly objects explicitly and the expensive cost of feature engineering procedure. Unlike existing appoaches which only partially solve these problems, we develop a unique framework t…
Exploring Multimodal Prompts For Unsupervised Continuous Anomaly Detection
Key Laboratory of Computing Power Network（计算能力网络重点实验室）；Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)（信息安全，教育部，山东计算机科学中心（济南国家超算中心），齐鲁工业大学（山东科学院））；Shandong Provincial Key Laboratory of Computing Power Internet（山东省计算能力互联网重点实验室）；Service Computing, Shandong Fundamental Research Center for Computer Science（服务计算，山东省计算机基础研究中心）；Faculty of Data Science, City University of Macau（数据科学学院，澳门城市大学）
Deep learning promises performant anomaly detection on time-variant datasets, but greatly suffers from low availability of suitable training datasets and frequently changing tasks. Deep transfer learning offers mitigation by letting algorithms built upon previous knowledge from different tasks or locations. In this article, a modular deep learning algorithm for anomaly detection on time series datasets is presented that allows for an easy integration of such transfer learning capabilities. It is thoroughly tested on a dataset from a discrete manufacturing process in order to prove its fundamen…
Anomaly Multi-classification in Industrial Scenarios: Transferring Few-shot Learning to a New Task
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（人机混合增强智能全国重点实验室）；National Engineering Research Center for Visual Information and Applications（视觉信息与应用国家工程研究中心）；Institute of Artificial Intelligence and Robotics（人工智能与机器人研究所）；Xi’an Jiaotong University（西安交通大学）；School of Informatics（信息学院）；Xiamen University（厦门大学）；College of Mechatronics and Control Engineering（机电与控制工程学院）；Shenzhen University（深圳大学）
本文提出工业异常多分类新任务，通过结合 RelationNet 与 PatchCore、伪类数据生成及对比学习，在 MvTec AD 和 MvTec3D AD 上取得优于直接微调 ResNet 的性能。
For a long time, anomaly localization has been widely used in industries. Previous studies focused on approximating the distribution of normal features without adaptation to a target dataset. However, since anomaly localization should precisely discriminate normal and abnormal features, the absence of adaptation may make the normality of abnormal features overestimated. Thus, we propose Coupled-hypersphere-based Feature Adaptation (CFA) which accomplishes sophisticated anomaly localization using features adapted to the target dataset. CFA consists of (1) a learnable patch descriptor that learn…
University of Ottawa（渥太华大学）；Queen’s University（女王大学）；National Research Council Canada（加拿大国家研究委员会）
In this paper, we develop and explore deep anomaly detection techniques based on the capsule network (CapsNet) for image data. Being able to encoding intrinsic spatial relationship between parts and a whole, CapsNet has been applied as both a classifier and deep autoencoder. This inspires us to design a prediction-probability-based and a reconstruction-error-based normality score functions for evaluating the "outlierness" of unseen images. Our results on three datasets demonstrate that the prediction-probability-based method performs consistently well, while the reconstruction-error-based appr…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Harvard University（哈佛大学）
In this paper, we propose Normality-Calibrated Autoencoder (NCAE), which can boost anomaly detection performance on the contaminated datasets without any prior information or explicit abnormal samples in the training phase. The NCAE adversarially generates high confident normal samples from a latent space having low entropy and leverages them to predict abnormal samples in a training dataset. NCAE is trained to minimise reconstruction errors in uncontaminated samples and maximise reconstruction errors in contaminated samples. The experimental results demonstrate that our method outperforms sha…
A Survey on Visual Anomaly Detection: Challenge, Approach, and Prospect
Huazhong University of Science and Technology（华中科技大学）；University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；Youtu Lab, Tencent（腾讯优图实验室）；Singapore Management University（新加坡管理大学）
Time series anomaly detection plays a critical role in automated monitoring systems. Most previous deep learning efforts related to time series anomaly detection were based on recurrent neural networks (RNN). In this paper, we propose a time series segmentation approach based on convolutional neural networks (CNN) for anomaly detection. Moreover, we propose a transfer learning framework that pretrains a model on a large-scale synthetic univariate time series data set and then fine-tunes its weights on small-scale, univariate or multivariate data sets with previously unseen classes of anomalies…
Nanjing University of Science and Technology（南京理工大学）；Singapore Management University（新加坡管理大学）；Nanyang Technological University（南洋理工大学）；Alibaba Group（阿里巴巴集团）
Weakly Supervised Video Anomaly Detection (WSVAD) is challenging because the binary anomaly label is only given on the video level, but the output requires snippet-level predictions. So, Multiple Instance Learning (MIL) is prevailing in WSVAD. However, MIL is notoriously known to suffer from many false alarms because the snippet-level detector is easily biased towards the abnormal snippets with simple context, confused by the normality with the same bias, and missing the anomaly with a different pattern. To this end, we propose a new MIL framework: Unbiased MIL (UMIL), to learn unbiased anomal…
Digital Sense（数字感知）；Universidad de la República（乌拉圭共和国大学）；IIE, Facultad de Ingeniería Universidad de la República（乌拉圭共和国大学工程学院IIE研究所）；Universidad Católica del Uruguay（乌拉圭天主教大学）
Anomalies can be defined as any non-random structure which deviates from normality. Anomaly detection methods reported in the literature are numerous and diverse, as what is considered anomalous usually varies depending on particular scenarios and applications. In this work we propose an a contrario framework to detect anomalies in images applying statistical analysis to feature maps obtained via convolutions. We evaluate filters learned from the image under analysis via patch PCA, Gabor filters and the feature maps obtained from a pre-trained deep neural network (Resnet). The proposed method …
In industrial applications, the early detection of malfunctioning factory machinery is crucial. In this paper, we consider acoustic malfunction detection via transfer learning. Contrary to the majority of current approaches which are based on deep autoencoders, we propose to extract features using neural networks that were pretrained on the task of image classification. We then use these features to train a variety of anomaly detection models and show that this improves results compared to convolutional autoencoders in recordings of four different factory machines in noisy environments. Moreov…
Digital Sense；Universidad de la República（共和国大学）；Universidad Católica del Uruguay（乌拉圭天主教大学）
Anomalies can be defined as any non-random structure which deviates from normality. Anomaly detection methods reported in the literature are numerous and diverse, as what is considered anomalous usually varies depending on particular scenarios and applications. In this work we propose an a contrario framework to detect anomalies in images applying statistical analysis to feature maps obtained via convolutions. We evaluate filters learned from the image under analysis via patch PCA, Gabor filters and the feature maps obtained from a pre-trained deep neural network (Resnet). The proposed method …
Exploring Intrinsic Normal Prototypes within a Single Image for Universal Anomaly Detection
Department of Precision Instrument, Tsinghua University（清华大学精密仪器系）；School of Mechanical Science & Engineering, Huazhong University of Science & Technology（华中科技大学机械科学与工程学院）
Calibrated One-class Classification for Unsupervised Time Series Anomaly Detection
National University of Defense Technology（国防科技大学）；College of Computer, National University of Defense Technology（国防科技大学计算机学院）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Singapore Management University（新加坡管理大学）
A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions
Centre for Artificial Intelligence（人工智能中心）；ZHAW School of Engineering（苏黎世应用科技大学工程学院）；Institute of Embedded Systems（嵌入式系统研究所）；Kistler Instrumente AG（奇石乐仪器股份公司）；Institute of Neuroinformatics（神经信息学研究所）；ETH & University of Zürich（苏黎世联邦理工学院与苏黎世大学）；ZHAW Centre for Artificial Intelligence（苏黎世应用科技大学人工智能中心）；European Centre for Living Technology (ECLT)（欧洲活技术中心（ECLT））
