A Survey on Visual Anomaly Detection: Challenge, Approach, and Prospect
Huazhong University of Science and Technology（华中科技大学）；University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；Youtu Lab, Tencent（腾讯优图实验室）；Singapore Management University（新加坡管理大学）
Visual Anomaly Detection (VAD) endeavors to pinpoint deviations from the concept of normality in visual data, widely applied across diverse domains, e.g., industrial defect inspection, and medical lesion detection. This survey comprehensively examines recent advancements in VAD by identifying three primary challenges: 1) scarcity of training data, 2) diversity of visual modalities, and 3) complexity of hierarchical anomalies. Starting with a brief overview of the VAD background and its generic concept definitions, we progressively categorize, emphasize, and discuss the latest VAD progress from the perspective of sample number, data modality, and anomaly hierarchy. Through an in-depth analysis of the VAD field, we finally summarize future developments for VAD and conclude the key findings and contributions of this survey.
University of Chinese Academy of Sciences（中国科学院大学）
Visual anomaly detection is an important and challenging problem in the field of machine learning and computer vision. This problem has attracted a considerable amount of attention in relevant research communities. Especially in recent years, the development of deep learning has sparked an increasing interest in the visual anomaly detection problem and brought a great variety of novel methods. In this paper, we provide a comprehensive survey of the classical and deep learning-based approaches for visual anomaly detection in the literature. We group the relevant approaches in view of their unde…
Southern University on Science and Technology（南方科技大学）；University of Surrey（萨里大学）；Tencent（腾讯）；Bielefeld University（比勒费尔德大学）
Visual sensory anomaly detection (AD) is an essential problem in computer vision, which is gaining momentum recently thanks to the development of AI for good. Compared with semantic anomaly detection which detects anomaly at the label level (semantic shift), visual sensory AD detects the abnormal part of the sample (covariate shift). However, no thorough review has been provided to summarize this area for the computer vision community. In this survey, we are the first one to provide a comprehensive review of visual sensory AD and category into three levels according to the form of anomalies. F…
Dalle Molle Institute for Artificial Intelligence (IDSIA), USI-SUPSI（达尔·莫莱人工智能研究所（IDSIA），提契诺大学-瑞士南部应用科学与艺术大学）
We consider the problem of building visual anomaly detection systems for mobile robots. Standard anomaly detection models are trained using large datasets composed only of non-anomalous data. However, in robotics applications, it is often the case that (potentially very few) examples of anomalies are available. We tackle the problem of exploiting these data to improve the performance of a Real-NVP anomaly detection model, by minimizing, jointly with the Real-NVP loss, an auxiliary outlier exposure margin loss. We perform quantitative experiments on a novel dataset (which we publish as suppleme…
Manipulated videos often contain subtle inconsistencies between their visual and audio signals. We propose a video forensics method, based on anomaly detection, that can identify these inconsistencies, and that can be trained solely using real, unlabeled data. We train an autoregressive model to generate sequences of audio-visual features, using feature sets that capture the temporal synchronization between video frames and sound. At test time, we then flag videos that the model assigns low probability. Despite being trained entirely on real videos, our model obtains strong performance on the …
Indian Institute of Technology Bhubaneswar（布巴内斯瓦尔印度理工学院）
Computer vision has evolved in the last decade as a key technology for numerous applications replacing human supervision. In this paper, we present a survey on relevant visual surveillance related researches for anomaly detection in public places, focusing primarily on roads. Firstly, we revisit the surveys done in the last 10 years in this field. Since the underlying building block of a typical anomaly detection is learning, we emphasize more on learning methods applied on video scenes. We then summarize the important contributions made during last six years on anomaly detection primarily foc…
Nanjing University of Science and Technology（南京理工大学）
Reconstruction method based on the memory module for visual anomaly detection attempts to narrow the reconstruction error for normal samples while enlarging it for anomalous samples. Unfortunately, the existing memory module is not fully applicable to the anomaly detection task, and the reconstruction error of the anomaly samples remains small. Towards this end, this work proposes a new unsupervised visual anomaly detection method to jointly learn effective normal features and eliminate unfavorable reconstruction errors. Specifically, a novel Partition Memory Bank (PMB) module is proposed to e…
Self-supervised learning allows for better utilization of unlabelled data. The feature representation obtained by self-supervision can be used in downstream tasks such as classification, object detection, segmentation, and anomaly detection. While classification, object detection, and segmentation have been investigated with self-supervised learning, anomaly detection needs more attention. We consider the problem of anomaly detection in images and videos, and present a new visual anomaly detection technique for videos. Numerous seminal and state-of-the-art self-supervised methods are evaluated…
VMAD: Visual-enhanced Multimodal Large Language Model for Zero-Shot Anomaly Detection
University of Science and Technology of China（中国科学技术大学）；Northeastern University（东北大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detection
School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机科学学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理学院计算与信息系统学院）；School of Artifical Intelligence, Xidian University（西安电子科技大学人工智能学院）
TD-VAD: Breaking Visual Dependence in Video Anomaly Detection with Text-Driven Learning
School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）；School of Artificial Intelligence Engineering, Hefei Institute of Technology（合肥工业大学人工智能工程学院）；China Mobile Zijin Innovation Institute（中国移动紫金创新研究院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）
AnomalyMoE: Towards a Language-free Generalist Model for Unified Visual Anomaly Detection
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China（中国科学院自动化研究所基础模型研究中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China（中国科学院大学人工智能学院）；Objecteye Inc., Beijing, China（Objecteye公司）
AF-CLIP: Zero-Shot Anomaly Detection via Anomaly-Focused CLIP Adaptation
School of Computer Science and Engineering（计算机科学与工程学院）；Sun Yat-sen University（中山大学）；Guangdong Key Laboratory of Big Data Analysis and Processing（大数据分析与处理广东省重点实验室）
VisionGPT: LLM-Assisted Real-Time Anomaly Detection for Safe Visual Navigation
School of Computing（计算机学院）；Clemson University（克莱姆森大学）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Stevens Institute of Technology（史蒂文斯理工学院）；School of Cultural and Social Studies（文化与社会研究学院）；Tokai University（东海大学）
Uncovering Anomalous Events for Marine Environmental Monitoring via Visual Anomaly Detection
Computer Science Department, IT University of Copenhagen（丹麦哥本哈根IT大学计算机科学系）；Anemo Robotics ApS（丹麦Anemo机器人公司）；Visual Analysis and Perception Laboratory, Aalborg University（丹麦奥胡斯大学视觉分析与感知实验室）；Pioneer Centre for Artificial Intelligence, Denmark（丹麦先锋人工智能中心）
Dalle Molle Institute for Artificial Intelligence (IDSIA), USI-SUPSI（达莱·莫勒人工智能研究所（IDSIA），提契诺大学-南方应用科学与艺术大学）
We consider the task of detecting anomalies for autonomous mobile robots based on vision. We categorize relevant types of visual anomalies and discuss how they can be detected by unsupervised deep learning methods. We propose a novel dataset built specifically for this task, on which we test a state-of-the-art approach; we finally discuss deployment in a real scenario.
Boosting Fine-Grained Visual Anomaly Detection with Coarse-Knowledge-Aware Adversarial Learning
Sun Yat-sen University（中山大学）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）；The Hong Kong Polytechnic University（香港理工大学）；Pazhou Lab（琶洲实验室）
Anomalies are by definition rare, thus labeled examples are very limited or nonexistent, and likely do not cover unforeseen scenarios. Unsupervised learning methods that don't necessarily encounter anomalies in training would be immensely useful. Generative vision models can be useful in this regard but do not sufficiently represent normal and abnormal data distributions. To this end, we propose constant curvature manifolds for embedding data distributions in unsupervised visual anomaly detection. Through theoretical and empirical explorations of manifold shapes, we develop a novel hyperspheri…
Tsinghua University（清华大学）；Huazhong University of Science and Technology（华中科技大学）
This paper presents a novel framework, named Global-Local Correspondence Framework (GLCF), for visual anomaly detection with logical constraints. Visual anomaly detection has become an active research area in various real-world applications, such as industrial anomaly detection and medical disease diagnosis. However, most existing methods focus on identifying local structural degeneration anomalies and often fail to detect high-level functional anomalies that involve logical constraints. To address this issue, we propose a two-branch approach that consists of a local branch for detecting struc…
The utilization of prior knowledge about anomalies is an essential issue for anomaly detections. Recently, the visual attention mechanism has become a promising way to improve the performance of CNNs for some computer vision tasks. In this paper, we propose a novel model called Layer-wise External Attention Network (LEA-Net) for efficient image anomaly detection. The core idea relies on the integration of unsupervised and supervised anomaly detectors via the visual attention mechanism. Our strategy is as follows: (i) Prior knowledge about anomalies is represented as the anomaly map generated b…
TokenCLIP: Token-wise Prompt Learning for Zero-shot Anomaly Detection
Zhejiang University（浙江大学）；Tencent YouTu Lab（腾讯YouTu实验室）；Singapore Management University（新加坡管理学院）；Qilu University of Technology (Shandong Academy of Sciences)（青岛科技大学（山东科学院））
IBM Research AI（IBM研究人工智能）；The University of Tokyo（东京大学）
Visual anomaly detection is common in several applications including medical screening and production quality check. Although a definition of the anomaly is an unknown trend in data, in many cases some hints or samples of the anomaly class can be given in advance. Conventional methods cannot use the available anomaly data, and also do not have a robustness of noise. In this paper, we propose a novel spatially-weighted reconstruction-loss-based anomaly detection with a likelihood value from a regression model trained by all known data. The spatial weights are calculated by a region of interest …
Tsinghua University（清华大学）；Huazhong University of Science and Technology（华中科技大学）
Industrial vision anomaly detection plays a critical role in the advanced intelligent manufacturing process, while some limitations still need to be addressed under such a context. First, existing reconstruction-based methods struggle with the identity mapping of trivial shortcuts where the reconstruction error gap is legible between the normal and abnormal samples, leading to inferior detection capabilities. Then, the previous studies mainly concentrated on the convolutional neural network (CNN) models that capture the local semantics of objects and neglect the global context, also resulting …
In this article, we evaluate unsupervised anomaly detection methods in multispectral images obtained with a wavelength-independent synthetic aperture sensing technique, called Airborne Optical Sectioning (AOS). With a focus on search and rescue missions that apply drones to locate missing or injured persons in dense forest and require real-time operation, we evaluate runtime vs. quality of these methods. Furthermore, we show that color anomaly detection methods that normally operate in the visual range always benefit from an additional far infrared (thermal) channel. We also show that, even wi…
