This paper presents a novel method that leverages a visual-language model, CLIP, as a data source for zero-shot anomaly detection. Tremendous efforts have been put towards developing anomaly detectors due to their potential industrial applications. Considering the difficulty in acquiring various anomalous samples for training, most existing methods train models with only normal samples and measure discrepancies from the distribution of normal samples during inference, which requires training a model for each object category. The problem of this inefficient training requirement has been tackled by designing a CLIP-based anomaly detector that applies prompt-guided classification to each part of an image in a sliding window manner. However, the method still suffers from the labor of careful prompt ensembling with known object categories. To overcome the issues above, we propose leveraging CLIP as a data source for training. Our method generates text embeddings with the text encoder in CLIP with typical prompts that include words of normal and anomaly. In addition to these words, we insert several randomly generated words into prompts, which enables the encoder to generate a diverse set of normal and anomalous samples. Using the generated embeddings as training data, a feed-forward neural network learns to extract features of normal and anomaly from CLIP's embeddings, and as a result, a category-agnostic anomaly detector can be obtained without any training images. Experimental results demonstrate that our method achieves state-of-the-art performance without laborious prompt ensembling in zero-shot setups.
AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIP
School of Biomedical Engineering, Division of Life Sciences and Medicine, USTC（中国科学技术大学生命科学与医学部生物医学工程学院）；MIRACLE Center, Suzhou Institute for Advance Research, USTC（中国科学技术大学苏州高等研究院MIRACLE中心）；Key Laboratory of Intelligent Information Processing of CAS, ICT, CAS（中国科学院计算技术研究所中国科学院智能信息处理重点实验室）；State Key Laboratory of Precision and Intelligent Chemistry, USTC（中国科学技术大学精准与智能化学国家重点实验室）；Stanford University（斯坦福大学）
AF-CLIP: Zero-Shot Anomaly Detection via Anomaly-Focused CLIP Adaptation
School of Computer Science and Engineering（计算机科学与工程学院）；Sun Yat-sen University（中山大学）；Guangdong Key Laboratory of Big Data Analysis and Processing（大数据分析与处理广东省重点实验室）
PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection
Harbin Engineering University（哈尔滨工程大学）；Key Laboratory of Advanced Marine Communication and Information Technology（先进海洋通信与信息技术重点实验室）；University of Toronto（多伦多大学）；Kanagawa University（神奈川大学）
CLIP3D-AD: Extending CLIP for 3D Few-Shot Anomaly Detection with Multi-View Images Generation
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人机混合增强智能全国重点实验室与人工智能与机器人研究所）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））；Xiamen University（厦门大学）；Shenzhen University（深圳大学）
Tsinghua University（清华大学）；New York University（纽约大学）
Video-based person re-identification has drawn massive attention in recent years due to its extensive applications in video surveillance. While deep learning-based methods have led to significant progress, these methods are limited by ineffectively using complementary information, which is blamed on necessary data augmentation in the training process. Data augmentation has been widely used to mitigate the over-fitting trap and improve the ability of network representation. However, the previous methods adopt image-based data augmentation scheme to individually process the input frames, which c…
MadCLIP: Few-shot Medical Anomaly Detection with CLIP
Department of Computer Science, University of Verona（威尼斯大学计算机科学系）；AI for Good (AIGO) Research Unit, Istituto Italiano di Tecnologia（意大利技术研究院人工智能为善研究单元）
GS-CLIP: Zero-shot 3D Anomaly Detection by Geometry-Aware Prompt and Synergistic View Representation Learning
School of Computer Science and Technology, Soochow University（苏州大学计算机科学与技术学院）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）
MADPOT: Medical Anomaly Detection with CLIP Adaptation and Partial Optimal Transport
Department of Computer Science, University of Verona（威尼斯大学计算机科学系）；AI for Good (AIGO) Research Unit, Istituto Italiano di Tecnologia（意大利技术研究院人工智能为善研究单元）
Enhancing Zero-Shot Anomaly Detection: CLIP-SAM Collaboration with Cascaded Prompts
School of Artificial Intelligence, Anhui University, Hefei, China（人工智能学院，安徽大学，合肥，中国）；Anhui Provincial Key Laboratory of Security Artificial Intelligence, Anhui University（安徽省安全人工智能重点实验室，安徽大学）；Anhui Provincial Engineering Research Center for Unmanned System（安徽省无人系统与智能技术工程研究中心）
本文提出一种用于工业异常检测的两阶段零样本异常分割框架，协同利用 CLIP 的异常定位与 SAM 的边界感知能力，通过 PPG 和 CPS 模块优化点提示与分割精度，在 Visa 等数据集上取得 SOTA 表现。
The proliferation of web platforms has created incentives for online abuse. Many graph-based anomaly detection techniques are proposed to identify the suspicious accounts and behaviors. However, most of them detect the anomalies once the users have performed many such behaviors. Their performance is substantially hindered when the users' observed data is limited at an early stage, which needs to be improved to minimize financial loss. In this work, we propose Eland, a novel framework that uses action sequence augmentation for early anomaly detection. Eland utilizes a sequence predictor to pred…
Bridging Vision Foundation Model Priors with CLIP for Spatial-aware Few-shot Anomaly Detection in Medical Images
The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）；Institute of Medical Intelligence and XR, The Chinese University of Hong Kong（香港中文大学医学智能与扩展现实研究所）
Deep neural networks are known to be vulnerable to unseen data: they may wrongly assign high confidence stcores to out-distribuion samples. Recent works try to solve the problem using representation learning methods and specific metrics. In this paper, we propose a simple, yet effective post-hoc anomaly detection algorithm named Test Time Augmentation Anomaly Detection (TTA-AD), inspired by a novel observation. Specifically, we observe that in-distribution data enjoy more consistent predictions for its original and augmented versions on a trained network than out-distribution data, which separ…
UltraAD: Fine-Grained Ultrasound Anomaly Classification via Few-Shot CLIP Adaptation
Computer Aided Medical Procedures (CAMP)（计算机辅助医疗程序）；TU Munich, Germany（慕尼黑工业大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；The First Affiliated Hospital of Sun Yat-Sen University（中山大学附属第一医院）
This paper proposes an autoencoder (AE) that is used for improving the performance of once-class classifiers for the purpose of detecting anomalies. Traditional one-class classifiers (OCCs) perform poorly under certain conditions such as high-dimensionality and sparsity. Also, the size of the training set plays an important role on the performance of one-class classifiers. Autoencoders have been widely used for obtaining useful latent variables from high-dimensional datasets. In the proposed approach, the AE is capable of deriving meaningful features from high-dimensional datasets while doing …
CLIP-Flow: A Universal Discriminator for AI-Generated Images Inspired by Anomaly Detection
Jilin University（吉林大学）；GIPSA-lab, Univ. Grenoble Alpes, CNRS, Grenoble INP（GIPSA实验室，格勒诺布尔阿尔卑斯大学，CNRS，格勒诺布尔INP）；Institute of Automation, Chinese Academy of Sciences（自动化研究所，中国科学院）
In this paper, we propose a weakly supervised deep temporal encoding-decoding solution for anomaly detection in surveillance videos using multiple instance learning. The proposed approach uses both abnormal and normal video clips during the training phase which is developed in the multiple instance framework where we treat video as a bag and video clips as instances in the bag. Our main contribution lies in the proposed novel approach to consider temporal relations between video instances. We deal with video instances (clips) as a sequential visual data rather than independent instances. We em…
ST Electronics - SUTD Cyber Security Laboratory（ST电子—新加坡科技设计大学网络安全实验室）；Singapore University of Technology and Design（新加坡科技设计大学）
Recently, the introduction of the generative adversarial network (GAN) and its variants has enabled the generation of realistic synthetic samples, which has been used for enlarging training sets. Previous work primarily focused on data augmentation for semi-supervised and supervised tasks. In this paper, we instead focus on unsupervised anomaly detection and propose a novel generative data augmentation framework optimized for this task. In particular, we propose to oversample infrequent normal samples - normal samples that occur with small probability, e.g., rare normal events. We show that th…
Macquarie University（麦考瑞大学）；University of Wollongong（伍伦贡大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Beihang University（北京航空航天大学）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）
Graph anomaly detection in this paper aims to distinguish abnormal nodes that behave differently from the benign ones accounting for the majority of graph-structured instances. Receiving increasing attention from both academia and industry, yet existing research on this task still suffers from two critical issues when learning informative anomalous behavior from graph data. For one thing, anomalies are usually hard to capture because of their subtle abnormal behavior and the shortage of background knowledge about them, which causes severe anomalous sample scarcity. Meanwhile, the overwhelming …
Anomaly detection in surveillance videos is an important research problem in computer vision. In this paper, we propose ADNet, an anomaly detection network, which utilizes temporal convolutions to localize anomalies in videos. The model works online by accepting consecutive windows consisting of fixed-number of video clips. Features extracted from video clips in a window are fed to ADNet, which allows to localize anomalies in videos effectively. We propose the AD Loss function to improve abnormal segment detection performance of ADNet. Additionally, we propose to use F1@k metric for temporal a…
Functional Localization Enforced Deep Anomaly Detection Using Fundus Images
Heinrich Heine University Düsseldorf（杜塞尔多夫海因里希·海涅大学）；Philipps University of Marburg（马尔堡菲利普斯大学）；University Hospital Knappschaftskrankenhaus Bochum（波鸿克纳普schaft医院大学医院）；University of Münster（明斯特大学）
Counterfactual Data Augmentation with Denoising Diffusion for Graph Anomaly Detection
School of Computer and Information Engineering, Henan University（河南大学计算机与信息工程学院）；University of Electronic Science and Technology of China（电子科技大学）；Sichuan University（四川大学）；Iowa State University（爱荷华州立大学）
DAS3D: Dual-modality Anomaly Synthesis for 3D Anomaly Detection
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Microsoft Research Asia（微软亚洲研究院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）
AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detection
School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机科学学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理学院计算与信息系统学院）；School of Artifical Intelligence, Xidian University（西安电子科技大学人工智能学院）
Imbalanced Graph-Level Anomaly Detection via Counterfactual Augmentation and Feature Learning
School of Cyber Science and Engineering, Wuhan University（武汉大学网络空间安全学院）；School of Computing, Macquarie University（麦考瑞大学计算学院）；Renmin Hospital of Wuhan University（武汉大学人民医院）
