Learning representations that clearly distinguish between normal and abnormal data is key to the success of anomaly detection. Most of existing anomaly detection algorithms use activation representations from forward propagation while not exploiting gradients from backpropagation to characterize data. Gradients capture model updates required to represent data. Anomalies require more drastic model updates to fully represent them compared to normal data. Hence, we propose the utilization of backpropagated gradients as representations to characterize model behavior on anomalies and, consequently, detect such anomalies. We show that the proposed method using gradient-based representations achieves state-of-the-art anomaly detection performance in benchmark image recognition datasets. Also, we highlight the computational efficiency and the simplicity of the proposed method in comparison with other state-of-the-art methods relying on adversarial networks or autoregressive models, which require at least 27 times more model parameters than the proposed method.
In this paper, we utilize weight gradients from backpropagation to characterize the representation space learned by deep learning algorithms. We demonstrate the utility of such gradients in applications including perceptual image quality assessment and out-of-distribution classification. The applications are chosen to validate the effectiveness of gradients as features when the test image distribution is distorted from the train image distribution. In both applications, the proposed gradient based features outperform activation features. In image quality assessment, the proposed approach is co…
A Unified Anomaly Synthesis Strategy with Gradient Ascent for Industrial Anomaly Detection and Localization
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；CASI Vision Technology CO., LTD.（CASI视觉技术有限公司）
Reconstruction from edge image combined with color and gradient difference for industrial surface anomaly detection
State Key Laboratory for Manufacturing System Engineering, Xi’an Jiaotong University（西安交通大学制造系统工程国家重点实验室）；International Joint Research Laboratory for Micro/Nano Manufacturing and Measurement Technologies, Xi’an Jiaotong University（西安交通大学微纳制造与测量技术国际联合研究实验室）
This brief sketches initial progress towards a unified energy-based solution for the semi-supervised visual anomaly detection and localization problem. In this setup, we have access to only anomaly-free training data and want to detect and identify anomalies of an arbitrary nature on test data. We employ the density estimates from the energy-based model (EBM) as normalcy scores that can be used to discriminate normal images from anomalous ones. Further, we back-propagate the gradients of the energy score with respect to the image in order to generate a gradient map that provides pixel-level sp…
Polytechnique Montréal（蒙特利尔高等理工学院）；Institute of Biomedical Engineering, Polytechnique Montréal（蒙特利尔高等理工学院生物医学工程研究所）
We propose a novel unsupervised approach based on a two-stage object-centric adversarial framework that only needs object regions for detecting frame-level local anomalies in videos. The first stage consists in learning the correspondence between the current appearance and past gradient images of objects in scenes deemed normal, allowing us to either generate the past gradient from current appearance or the reverse. The second stage extracts the partial reconstruction errors between real and generated images (appearance and past gradient) with normal object behaviour, and trains a discriminato…
This study presents an open source method for detecting 3D printing anomalies by comparing images of printed layers from a stationary monocular camera with G-code-based reference images of an ideal process generated with Blender, a physics rendering engine. Recognition of visual deviations was accomplished by analyzing the similarity of histograms of oriented gradients (HOG) of local image areas. The developed technique requires preliminary modeling of the working environment to achieve the best match for orientation, color rendering, lighting, and other parameters of the printed part. The out…
Hyperspectral Anomaly Detection Fused Unified Nonconvex Tensor Ring Factors Regularization
School of Mathematics and Statistics, Southwest University（数学与统计学学院，西南大学）；School of Mathematics and Statistics, Xi’an Jiaotong University（数学与统计学学院，西安交通大学）；School of Computer Science and Technology and Ministry of Education Key Lab For Intelligent Networks and Network Security, Xi’an Jiaotong University（计算机科学与技术学院和教育部智能网络与网络安全重点实验室，西安交通大学）；School of Mathematical Sciences/Research Center for Image and Vision Computing, University of Electronic Science and Technology of China（数学科学学院/图像与视觉计算研究中心，电子科技大学）；Institute of Methodologies for Environmental Analysis, CNR-IMAA（环境分析方法学研究所，CNR-IMAA）
The Hong Kong University of Science and Technology（香港科技大学）；Tencent AI Lab（腾讯AI实验室）；Imsight AI Research Lab（依图AI研究院）；Xiamen University（厦门大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深圳－香港产学研协同创新研究院）
In this paper, we propose a model-based characterization of neural networks to detect novel input types and conditions. Novelty detection is crucial to identify abnormal inputs that can significantly degrade the performance of machine learning algorithms. Majority of existing studies have focused on activation-based representations to detect abnormal inputs, which limits the characterization of abnormality from a data perspective. However, a model perspective can also be informative in terms of the novelties and abnormalities. To articulate the significance of the model perspective in novelty …
One-for-More: Continual Diffusion Model for Anomaly Detection
East China Normal University（东华师范大学）；Shanghai Innovation Institute（上海创新研究院）；Xiamen University（厦门大学）；Shanghai Jiao Tong University（上海交通大学）；Tencent YouTu Lab（腾讯YouTu实验室）；Tencent WeChatPay Lab33（腾讯微信支付实验室）；CATL（宁德时代）；Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China（教育部多媒体可信感知与高效计算重点实验室）
本文针对扩散模型在持续异常检测中的保真幻觉与灾难性遗忘问题，提出基于梯度投影的持续扩散模型，并通过迭代奇异值分解降低内存开销，同时引入异常掩码网络增强条件机制，在 MVTec 和 VisA 上取得领先性能。
National Taiwan University（台湾大学）；National Tsing Hua University（清华大学）
Anomaly awareness is an essential capability for safety-critical applications such as autonomous driving. While recent progress of robotics and computer vision has enabled anomaly detection for image classification, anomaly detection on semantic segmentation is less explored. Conventional anomaly-aware systems assuming other existing classes as out-of-distribution (pseudo-unknown) classes for training a model will result in two drawbacks. (1) Unknown classes, which applications need to cope with, might not actually exist during training time. (2) Model performance would strongly rely on the cl…
Variational Auto-Encoders (VAEs) have shown great potential in the unsupervised learning of data distributions. An VAE trained on normal images is expected to only be able to reconstruct normal images, allowing the localization of anomalous pixels in an image via manipulating information within the VAE ELBO loss. The ELBO consists of KL divergence loss (image-wise) and reconstruction loss (pixel-wise). It is natural and straightforward to use the later as the predictor. However, usually local anomaly added to a normal image can deteriorate the whole reconstructed image, causing segmentation us…
Unicom (Shanghai) Industry Internet Co., Ltd.（联通（上海）产业互联网有限公司）
The unsupervised anomaly localization task faces the challenge of missing anomaly sample training, detecting multiple types of anomalies, and dealing with the proportion of the area of multiple anomalies. A separate teacher-student feature imitation network structure and a multi-scale processing strategy combining an image and feature pyramid are proposed to solve these problems. A network module importance search method based on gradient descent optimization is proposed to simplify the network structure. The experimental results show that the proposed algorithm performs better than the featur…
Autoencoder reconstructions are widely used for the task of unsupervised anomaly localization. Indeed, an autoencoder trained on normal data is expected to only be able to reconstruct normal features of the data, allowing the segmentation of anomalous pixels in an image via a simple comparison between the image and its autoencoder reconstruction. In practice however, local defects added to a normal image can deteriorate the whole reconstruction, making this segmentation challenging. To tackle the issue, we propose in this paper a new approach for projecting anomalous data on a autoencoder-lear…
We propose to utilize gradients for detecting adversarial and out-of-distribution samples. We introduce confounding labels -- labels that differ from normal labels seen during training -- in gradient generation to probe the effective expressivity of neural networks. Gradients depict the amount of change required for a model to properly represent given inputs, providing insight into the representational power of the model established by network architectural properties as well as training data. By introducing a label of different design, we remove the dependency on ground truth labels for gradi…
