The detection and the quantification of anomalies in image data are critical tasks in industrial scenes such as detecting micro scratches on product. In recent years, due to the difficulty of defining anomalies and the limit of correcting their labels, research on unsupervised anomaly detection using generative models has attracted attention. Generally, in those studies, only normal images are used for training to model the distribution of normal images. The model measures the anomalies in the target images by reproducing the most similar images and scoring image patches indicating their fit to the learned distribution. This approach is based on a strong presumption; the trained model should not be able to generate abnormal images. However, in reality, the model can generate abnormal images mainly due to noisy normal data which include small abnormal pixels, and such noise severely affects the accuracy of the model. Therefore, we propose a novel anomaly detection method to distort the distribution of the model with existing abnormal images. The proposed method detects pixel-level micro anomalies with a high accuracy from 1024x1024 high resolution images which are actually used in an industrial scene. In this paper, we share experimental results on open datasets, due to the confidentiality of the data.
École Polytechnique（巴黎综合理工学院）；University of California, Berkeley（加州大学伯克利分校）；Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）
Anomaly detection has been an active research area with a wide range of potential applications. Key challenges for anomaly detection in the AI era with big data include lack of prior knowledge of potential anomaly types, highly complex and noisy background in input data, scarce abnormal samples, and imbalanced training dataset. In this work, we propose a meta-learning framework for anomaly detection to deal with these issues. Within this framework, we incorporate the idea of generative adversarial networks (GANs) with appropriate choices of loss functions including structural similarity index …
Key Laboratory of Multimedia Trusted Perception and Efficient Computing（多媒体可信感知与高效计算重点实验室）；Ministry of Education of China（中华人民共和国教育部）；Xiamen University（厦门大学）；VIVO
In medical imaging, obtaining large amounts of labeled data is often a hurdle, because annotations and pathologies are scarce. Anomaly detection is a method that is capable of detecting unseen abnormal data while only being trained on normal (unannotated) data. Several algorithms based on generative adversarial networks (GANs) exist to perform this task, yet certain limitations are in place because of the instability of GANs. This paper proposes a new method by combining an existing method, GANomaly, with progressively growing GANs. The latter is known to be more stable, considering its abilit…
JR East Information Systems Company（东日本旅客铁道信息系统公司）；East Japan Railway Company（东日本旅客铁道公司）
The external visual inspections of rolling stock's underfloor equipment are currently being performed via human visual inspection. In this study, we attempt to partly automate visual inspection by investigating anomaly inspection algorithms that use image processing technology. As the railroad maintenance studies tend to have little anomaly data, unsupervised learning methods are usually preferred for anomaly detection; however, training cost and accuracy is still a challenge. Additionally, a researcher created anomalous images from normal images by adding noise, etc., but the anomalous target…
Generative Adversarial Networks (GANs) have been used extensively and quite successfully for unsupervised learning. As GANs don't approximate an explicit probability distribution, it's an interesting study to inspect the latent space representations learned by GANs. The current work seeks to push the boundaries of such inspection methods to further understand in more detail the manifold being learned by GANs. Various interpolation and extrapolation techniques along with vector arithmetic is used to understand the learned manifold. We show through experiments that GANs indeed learn a data proba…
Tsinghua University（清华大学）；University of Cincinnati（辛辛那提大学）
Visual surface inspection is a challenging task owing to the highly diverse appearance of target surfaces and defective regions. Previous attempts heavily rely on vast quantities of training examples with manual annotation. However, in some practical cases, it is difficult to obtain a large number of samples for inspection. To combat it, we propose a hierarchical texture-perceiving generative adversarial network (HTP-GAN) that is learned from the one-shot normal image in an unsupervised scheme. Specifically, the HTP-GAN contains a pyramid of convolutional GANs that can capture the global struc…
Information Science and Technology Institute (ISTI) of CNR（意大利国家研究委员会信息科学与技术研究所（ISTI））
In this work, we propose CBiGAN -- a novel method for anomaly detection in images, where a consistency constraint is introduced as a regularization term in both the encoder and decoder of a BiGAN. Our model exhibits fairly good modeling power and reconstruction consistency capability. We evaluate the proposed method on MVTec AD -- a real-world benchmark for unsupervised anomaly detection on high-resolution images -- and compare against standard baselines and state-of-the-art approaches. Experiments show that the proposed method improves the performance of BiGAN formulations by a large margin a…
Quality inspection applications in industry are required to move towards a zero-defect manufacturing scenario, withnon-destructive inspection and traceability of 100 % of produced parts. Developing robust fault detection and classification modelsfrom the start-up of the lines is challenging due to the difficulty in getting enough representative samples of the faulty patternsand the need to manually label them. This work presents a methodology to develop a robust inspection system, targeting thesepeculiarities, in the context of solar cell manufacturing. The methodology is divided into two phas…
Anomaly detection is a difficult problem in many areas and has recently been subject to a lot of attention. Classifying unseen data as anomalous is a challenging matter. Latest proposed methods rely on Generative Adversarial Networks (GANs) to estimate the normal data distribution, and produce an anomaly score prediction for any given data. In this article, we propose a simple yet new adversarial method to tackle this problem, denoted as History-based anomaly detector (HistoryAD). It consists of a self-supervised model, trained to recognize 'normal' samples by comparing them to samples based o…
Universidade Federal de Minas Gerais（米纳斯吉拉斯联邦大学）；Centro Federal de Educação Tecnológica de Minas Gerais（米纳斯吉拉斯联邦教育技术中心）
Iterating on creating pixel art character sprite sheets is essential to the game development process. However, it can take a lot of effort until the final versions containing different poses and animation clips are achieved. This paper investigates using conditional generative adversarial networks to aid the designers in creating such sprite sheets. We propose an architecture based on Pix2Pix to generate images of characters facing a target side (e.g., right) given sprites of them in a source pose (e.g., front). Experiments with small pixel art datasets yielded promising results, resulting in …
Gina Cody School of Engineering and Computer Science（吉娜·科迪工程与计算机科学学院）；Concordia University（康考迪亚大学）
Identifying anomalies refers to detecting samples that do not resemble the training data distribution. Many generative models have been used to find anomalies, and among them, generative adversarial network (GAN)-based approaches are currently very popular. GANs mainly rely on the rich contextual information of these models to identify the actual training distribution. Following this analogy, we suggested a new unsupervised model based on GANs --a combination of an autoencoder and a GAN. Further, a new scoring function was introduced to target anomalies where a linear combination of the intern…
Underwater Vehicles have become more sophisticated, driven by the off-shore sector and the scientific community's rapid advancements in underwater operations. Notably, many underwater tasks, including the assessment of subsea infrastructure, are performed with the assistance of Autonomous Underwater Vehicles (AUVs). There have been recent breakthroughs in Artificial Intelligence (AI) and, notably, Deep Learning (DL) models and applications, which have widespread usage in a variety of fields, including aerial unmanned vehicles, autonomous car navigation, and other applications. However, they ar…
Generative adversarial networks (GANs) are one of the most popular methods for generating images today. While impressive results have been validated by visual inspection, a number of quantitative criteria have emerged only recently. We argue here that the existing ones are insufficient and need to be in adequation with the task at hand. In this paper we introduce two measures based on image classification---GAN-train and GAN-test, which approximate the recall (diversity) and precision (quality of the image) of GANs respectively. We evaluate a number of recent GAN approaches based on these two …
Bosch Center for AI（博世人工智能中心）；MPI for Informatics（马克斯·普朗克信息学研究所）；University of Tübingen（图宾根大学）
Prior work has extensively studied the latent space structure of GANs for unconditional image synthesis, enabling global editing of generated images by the unsupervised discovery of interpretable latent directions. However, the discovery of latent directions for conditional GANs for semantic image synthesis (SIS) has remained unexplored. In this work, we specifically focus on addressing this gap. We propose a novel optimization method for finding spatially disentangled class-specific directions in the latent space of pretrained SIS models. We show that the latent directions found by our method…
University of Genoa（热那亚大学）；Queen Mary University of London（伦敦玛丽女王大学）；Carlos III University of Madrid（马德里卡洛斯三世大学）
This paper focuses on multi-sensor anomaly detection for moving cognitive agents using both external and private first-person visual observations. Both observation types are used to characterize agents' motion in a given environment. The proposed method generates locally uniform motion models by dividing a Gaussian process that approximates agents' displacements on the scene and provides a Shared Level (SL) self-awareness based on Environment Centered (EC) models. Such models are then used to train in a semi-unsupervised way a set of Generative Adversarial Networks (GANs) that produce an estim…
Fraunhofer Center for Machine Learning（弗劳恩霍夫机器学习中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Fraunhofer Institute for Optronics, System Technologies and Image Exploitation IOSB（弗劳恩霍夫光电子、系统技术与图像利用研究所（IOSB））
Anomaly detection plays in many fields of research, along with the strongly related task of outlier detection, a very important role. Especially within the context of the automated analysis of video material recorded by surveillance cameras, abnormal situations can be of very different nature. For this purpose this work investigates Generative-Adversarial-Network-based methods (GAN) for anomaly detection related to surveillance applications. The focus is on the usage of static camera setups, since this kind of camera is one of the most often used and belongs to the lower price segment. In orde…
University of New South Wales（新南威尔士大学）；Princess Nourah bint Abdulrahman University（诺拉·本特·阿卜杜勒拉赫曼公主大学）
With the increasing interest in the content creation field in multiple sectors such as media, education, and entertainment, there is an increasing trend in the papers that uses AI algorithms to generate content such as images, videos, audio, and text. Generative Adversarial Networks (GANs) in one of the promising models that synthesizes data samples that are similar to real data samples. While the variations of GANs models, in general, have been covered to some extent in several survey papers, to the best of our knowledge, this is among the first survey papers that reviews the state-of-the-art…
For steel product manufacturing in indoor factories, steel defect detection is important for quality control. For example, a steel sheet is extremely delicate, and must be accurately inspected. However, to maintain the painted steel parts of the infrastructure around a severe outdoor environment, corrosion detection is critical for predictive maintenance. In this paper, we propose a general-purpose application for steel anomaly detection that consists of the following four components. The first, a learner, is a unit image classification network to determine whether the region of interest or ba…
Shenzhen University（深圳大学）；Guangzhou Medical University（广州医科大学）
Ultrasound (US) is widely accepted in clinic for anatomical structure inspection. However, lacking in resources to practice US scan, novices often struggle to learn the operation skills. Also, in the deep learning era, automated US image analysis is limited by the lack of annotated samples. Efficiently synthesizing realistic, editable and high resolution US images can solve the problems. The task is challenging and previous methods can only partially complete it. In this paper, we devise a new framework for US image synthesis. Particularly, we firstly adopt a sketch generative adversarial netw…
SenseTime Research（商汤科技研究院）；JD.com Inc Qing Yuan Research Institute（京东集团清源研究院）；Shanghai Jiao Tong University（上海交通大学）
Visual anomaly detection plays a crucial role in not only manufacturing inspection to find defects of products during manufacturing processes, but also maintenance inspection to keep equipment in optimum working condition particularly outdoors. Due to the scarcity of the defective samples, unsupervised anomaly detection has attracted great attention in recent years. However, existing datasets for unsupervised anomaly detection are biased towards manufacturing inspection, not considering maintenance inspection which is usually conducted under outdoor uncontrolled environment such as varying cam…
Max Planck Institute for Intelligence Systems（马克斯·普朗克智能系统研究所）；University Carlos III in Madrid（马德里卡洛斯三世大学）；Swiss Data Science Center (ETH Zürich)（瑞士数据科学中心（苏黎世联邦理工学院））
We propose a novel training procedure for improving the performance of generative adversarial networks (GANs), especially to bidirectional GANs. First, we enforce that the empirical distribution of the inverse inference network matches the prior distribution, which favors the generator network reproducibility on the seen samples. Second, we have found that the marginal log-likelihood of the samples shows a severe overrepresentation of a certain type of samples. To address this issue, we propose to train the bidirectional GAN using a non-uniform sampling for the mini-batch selection, resulting …
Corporate Research and Development Center, Toshiba Corporation（东芝公司企业研发中心）
Generative adversarial networks (GANs) have shown promise for various problems including anomaly detection. When anomaly detection is performed using GAN models that learn only the features of normal data samples, data that are not similar to normal data are detected as abnormal samples. The present approach is developed by employing a dual-encoder in a bidirectional GAN architecture that is trained simultaneously with a generator and a discriminator network. Through the learning mechanism, the proposed method aims to reduce the problem of bad cycle consistency, in which a bidirectional GAN mi…
Pancreatic Tumor Segmentation as Anomaly Detection in CT Images Using Denoising Diffusion Models
School of Electrical and Computer Engineering, University of Oklahoma（俄克拉荷马大学电气与计算机工程学院）；University of Oklahoma Health Sciences Center（俄克拉荷马大学健康科学中心）；School of Computer Science, University of Oklahoma（俄克拉荷马大学计算机科学学院）
Unsupervised learning of anomaly detection in high-dimensional data, such as images, is a challenging problem recently subject to intense research. Through careful modelling of the data distribution of normal samples, it is possible to detect deviant samples, so called anomalies. Generative Adversarial Networks (GANs) can model the highly complex, high-dimensional data distribution of normal image samples, and have shown to be a suitable approach to the problem. Previously published GAN-based anomaly detection methods often assume that anomaly-free data is available for training. However, this…
Attention Modules Improve Image-Level Anomaly Detection for Industrial Inspection: A DifferNet Case Study
Voxar Labs, Centro de Informática, Universidade Federal de Pernambuco（佩南布科联邦大学信息学中心 Voxar 实验室）；Visual Computing Lab, DC, Universidade Federal Rural de Pernambuco（佩南布科联邦农村大学 DC 视觉计算实验室）；Chemnitz University of Technology（开姆尼茨工业大学）
Digital Sense；Universidad de la República（共和国大学）；Universidad Católica del Uruguay（乌拉圭天主教大学）
Anomalies can be defined as any non-random structure which deviates from normality. Anomaly detection methods reported in the literature are numerous and diverse, as what is considered anomalous usually varies depending on particular scenarios and applications. In this work we propose an a contrario framework to detect anomalies in images applying statistical analysis to feature maps obtained via convolutions. We evaluate filters learned from the image under analysis via patch PCA, Gabor filters and the feature maps obtained from a pre-trained deep neural network (Resnet). The proposed method …
Machine learning (ML) algorithms are optimized for the distribution represented by the training data. For outlier data, they often deliver predictions with equal confidence, even though these should not be trusted. In order to deploy ML-based digital pathology solutions in clinical practice, effective methods for detecting anomalous data are crucial to avoid incorrect decisions in the outlier scenario. We propose a new unsupervised learning approach for anomaly detection in histopathology data based on generative adversarial networks (GANs). Compared to the existing GAN-based methods that have…
Electroplated diamond abrasive tools require nickel coating on a metal surface for abrasive bonding and part functionality. The electroplated nickel-coated abrasive tool is expected to have a high-quality part performance by having a nickel coating thickness of between 50% to 60% of the abrasive median diameter, uniformity of the nickel layer, abrasive distribution over the electroplated surface, and bright gloss. Electroplating parameters are set accordingly for this purpose. Industrial quality inspection for defects of these abrasive electroplated parts with optical inspection instruments is…
University of Chinese Academy of Sciences（中国科学院大学）；Beijing Technology and Business University（北京工商大学）；Indian Institute of Technology Patna（印度理工学院巴特那分校）
Currently, deep learning-based visual inspection has been highly successful with the help of supervised learning methods. However, in real industrial scenarios, the scarcity of defect samples, the cost of annotation, and the lack of a priori knowledge of defects may render supervised-based methods ineffective. In recent years, unsupervised anomaly localization algorithms have become more widely used in industrial inspection tasks. This paper aims to help researchers in this field by comprehensively surveying recent achievements in unsupervised anomaly localization in industrial images using de…
Automatic detection of visual anomalies and changes in the environment has been a topic of recurrent attention in the fields of machine learning and computer vision over the past decades. A visual anomaly or change detection algorithm identifies regions of an image that differ from a reference image or dataset. The majority of existing approaches focus on anomaly or fault detection in a specific class of images or environments, while general purpose visual anomaly detection algorithms are more scarce in the literature. In this paper, we propose a comprehensive deep learning framework for detec…
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen Bwell Technology Co., Ltd（深圳博铭维技术有限公司）；Shenzhen Longhua Drainage Co., Ltd（深圳市龙华排水有限公司）；Shanghai AI Laboratory（上海人工智能实验室）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）
Video understanding is an important problem in computer vision. Currently, the well-studied task in this research is human action recognition, where the clips are manually trimmed from the long videos, and a single class of human action is assumed for each clip. However, we may face more complicated scenarios in the industrial applications. For example, in the real-world urban pipe system, anomaly defects are fine-grained, multi-labeled, domain-relevant. To recognize them correctly, we need to understand the detailed video content. For this reason, we propose to advance research areas of video…
Attention Modules Improve Modern Image-Level Anomaly Detection: A DifferNet Case Study
Voxar Labs, Centro de Informática, Universidade Federal de Pernambuco（伯南布哥联邦大学信息中心Voxar实验室）；Visual Computing Lab, DC, Universidade Federal Rural de Pernambuco（伯南布哥联邦农村大学视觉计算实验室）；Junior Professorship of Media Computing, Chemnitz University of Technology（开姆尼茨工业大学媒体计算初级教授职位）
Within commercial wind energy generation, the monitoring and predictive maintenance of wind turbine blades in-situ is a crucial task, for which remote monitoring via aerial survey from an Unmanned Aerial Vehicle (UAV) is commonplace. Turbine blades are susceptible to both operational and weather-based damage over time, reducing the energy efficiency output of turbines. In this study, we address automating the otherwise time-consuming task of both blade detection and extraction, together with fault detection within UAV-captured turbine blade inspection imagery. We propose BladeNet, an applicati…
A 3D Multimodal Feature for Infrastructure Anomaly Detection
Department of Engineering Science, University of Oxford（工程科学系，牛津大学）；Department of Geotechnical Engineering, College of Civil Engineering, Tongji University（地质工程系，同济大学土木学院）；Construction Engineering, University of Cambridge（建设工程，剑桥大学）
