Robust autonomous driving requires agents to accurately identify unexpected areas (anomalies) in urban scenes. To this end, some critical issues remain open: how to design advisable metric to measure anomalies, and how to properly generate training samples of anomaly data? Classical effort in anomaly detection usually resorts to pixel-wise uncertainty or sample synthesis, which ignores the contextual information and sometimes requires auxiliary data with fine-grained annotations. On the contrary, in this paper, we exploit the strong context-dependent nature of the segmentation task and design an energy-guided self-supervised framework for anomaly segmentation, which optimizes an anomaly head by maximizing the likelihood of self-generated anomaly pixels. For this purpose, we design two estimators to model anomaly likelihood, one is a task-agnostic binary estimator and the other depicts the likelihood as residual of task-oriented joint energy. Based on the proposed estimators, we devise an adaptive self-supervised training framework, which exploits the contextual reliance and estimated likelihood to refine mask annotations in anomaly areas. We conduct extensive experiments on challenging Fishyscapes and Road Anomaly benchmarks, demonstrating that without any auxiliary data or synthetic models, our method can still achieve comparable performance to supervised competitors. Code is available at https://github.com/yuanpengtu/SLEEG..
Australian Institute for Machine Learning（澳大利亚机器学习研究所）；University of Adelaide（阿德莱德大学）；Singapore Management University（新加坡管理大学）
State-of-the-art (SOTA) anomaly segmentation approaches on complex urban driving scenes explore pixel-wise classification uncertainty learned from outlier exposure, or external reconstruction models. However, previous uncertainty approaches that directly associate high uncertainty to anomaly may sometimes lead to incorrect anomaly predictions, and external reconstruction models tend to be too inefficient for real-time self-driving embedded systems. In this paper, we propose a new anomaly segmentation method, named pixel-wise energy-biased abstention learning (PEBAL), that explores pixel-wise a…
Anomaly segmentation is a crucial task for safety-critical applications, such as autonomous driving in urban scenes, where the goal is to detect out-of-distribution (OOD) objects with categories which are unseen during training. The core challenge of this task is how to distinguish hard in-distribution samples from OOD samples, which has not been explicitly discussed yet. In this paper, we propose a novel and simple approach named Consensus Synergizes with Memory (CosMe) to address this challenge, inspired by the psychology finding that groups perform better than individuals on memory tasks. T…
Learning to drive faithfully in highly stochastic urban settings remains an open problem. To that end, we propose a Multi-task Learning from Demonstration (MT-LfD) framework which uses supervised auxiliary task prediction to guide the main task of predicting the driving commands. Our framework involves an end-to-end trainable network for imitating the expert demonstrator's driving commands. The network intermediately predicts visual affordances and action primitives through direct supervision which provide the aforementioned auxiliary supervised guidance. We demonstrate that such joint learnin…
Weakly supervised video anomaly detection (WSVAD) is a challenging task since only video-level labels are available for training. In previous studies, the discriminative power of the learned features is not strong enough, and the data imbalance resulting from the mini-batch training strategy is ignored. To address these two issues, we propose a novel WSVAD method based on cross-batch clustering guidance. To enhance the discriminative power of features, we propose a batch clustering based loss to encourage a clustering branch to generate distinct normal and abnormal clusters based on a batch of…
SARD: Segmentation-Aware Anomaly Synthesis via Region-Constrained Diffusion with Discriminative Mask Guidance
Global Institute of Future Technology（未来技术全球研究所）；Shanghai Jiao Tong University（上海交通大学）；Department of Intelligent Manufacturing（智能制造系）；Contemporary Amperex Technology Co.,Ltd（当代电子技术有限公司）
For autonomous driving, radar is an important sensor type. On the one hand, radar offers a direct measurement of the radial velocity of targets in the environment. On the other hand, in literature, radar sensors are known for their robustness against several kinds of adverse weather conditions. However, on the downside, radar is susceptible to ghost targets or clutter which can be caused by several different causes, e.g., reflective surfaces in the environment. Ghost targets, for instance, can result in erroneous object detections. To this end, it is desirable to identify anomalous targets as …
Physics-inspired Pseudo Anomaly Generation and Prototype Feature Guidance for 3D Anomaly Detection
School of Computer Science, Wuhan University（武汉大学计算机科学学院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））；School of Information and Software Engineering, East China Jiaotong University（华东交通大学信息工程学院）；School of Computer Science and Technology, Dalian University of Technology（大连理工大学计算机科学与技术学院）
Nanjing University of Science and Technology（南京理工大学）
Unsupervised anomaly detection is a challenging task in industrial applications since it is impracticable to collect sufficient anomalous samples. In this paper, a novel Self-Supervised Guided Segmentation Framework (SGSF) is proposed by jointly exploring effective generation method of forged anomalous samples and the normal sample features as the guidance information of segmentation for anomaly detection. Specifically, to ensure that the generated forged anomaly samples are conducive to model training, the Saliency Augmentation Module (SAM) is proposed. SAM introduces a saliency map to genera…
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen Bwell Technology Co., Ltd（深圳博铭维技术有限公司）；Shenzhen Longhua Drainage Co., Ltd（深圳市龙华排水有限公司）；Shanghai AI Laboratory（上海人工智能实验室）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）
Video understanding is an important problem in computer vision. Currently, the well-studied task in this research is human action recognition, where the clips are manually trimmed from the long videos, and a single class of human action is assumed for each clip. However, we may face more complicated scenarios in the industrial applications. For example, in the real-world urban pipe system, anomaly defects are fine-grained, multi-labeled, domain-relevant. To recognize them correctly, we need to understand the detailed video content. For this reason, we propose to advance research areas of video…
MAGIC: Few-Shot Mask-Guided Anomaly Inpainting with Prompt Perturbation, Spatially Adaptive Guidance, and Context Awareness
Department of AI Semiconductor Engineering, Hanyang University, Seoul, Korea（汉阳大学人工智能半导体工程系）；Department of Electronic Engineering, Hanyang University, Seoul, Korea（汉阳大学电子工程系）
Enhancing Weakly Supervised Multimodal Video Anomaly Detection through Text Guidance
School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）；Alibaba Cloud（阿里云）；College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）
In medical applications, weakly supervised anomaly detection methods are of great interest, as only image-level annotations are required for training. Current anomaly detection methods mainly rely on generative adversarial networks or autoencoder models. Those models are often complicated to train or have difficulties to preserve fine details in the image. We present a novel weakly supervised anomaly detection method based on denoising diffusion implicit models. We combine the deterministic iterative noising and denoising scheme with classifier guidance for image-to-image translation between d…
Manipal Institute of Technology（马尼帕尔理工学院）；Manipal Academy of Higher Education（马尼帕尔高等教育学院）；Manipal Institute of Technology Bengaluru（马尼帕尔理工学院班加罗尔分校）
UAV based surveillance is gaining much interest worldwide due to its extensive applications in monitoring wildlife, urban planning, disaster management, campus security, etc. These videos are analyzed for strange/odd/anomalous patterns which are essential aspects of surveillance. But manual analysis of these videos is tedious and laborious. Hence, the development of computer-aided systems for the analysis of UAV based surveillance videos is crucial. Despite this interest, in literature, several computer aided systems are developed focusing only on CCTV based surveillance videos. These methods …
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）；Institute of Robotics and Intelligent Manufacturing, Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）机器人与智能制造研究院）
As a kind of generative self-supervised learning methods, generative adversarial nets have been widely studied in the field of anomaly detection. However, the representation learning ability of the generator is limited since it pays too much attention to pixel-level details, and generator is difficult to learn abstract semantic representations from label prediction pretext tasks as effective as discriminator. In order to improve the representation learning ability of generator, we propose a self-supervised learning framework combining generative methods and discriminative methods. The generato…
Zhejiang University（浙江大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Adobe Research（奥多比研究中心）；Shanghai AI Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）
Purely MLP-based neural radiance fields (NeRF-based methods) often suffer from underfitting with blurred renderings on large-scale scenes due to limited model capacity. Recent approaches propose to geographically divide the scene and adopt multiple sub-NeRFs to model each region individually, leading to linear scale-up in training costs and the number of sub-NeRFs as the scene expands. An alternative solution is to use a feature grid representation, which is computationally efficient and can naturally scale to a large scene with increased grid resolutions. However, the feature grid tends to be…
Double Helix Diffusion for Cross-Domain Anomaly Image Generation
School of Computer Science, Wuhan University（武汉大学计算机学院）；Shenzhen Intellifusion Technologies Co Ltd（深圳智融科技有限公司）；Guangdong Artificial Intelligence and Digital Economy Laboratory (SZ)（广东省人工智能与数字经济实验室）
Reservoir-enhanced Segment Anything Model for Subsurface Diagnosis
School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；Research Institute of Mine Artificial Intelligence, Chinese Institute of Coal Science（中国煤炭科学研究院矿山人工智能研究院）
Urban Flood Mapping Using Satellite Synthetic Aperture Radar Data: A Review of Characteristics, Approaches and Datasets
Technical University of Munich（慕尼黑工业大学）；Vienna University of Technology（维也纳工业大学）；ETH Zurich（苏黎世联邦理工学院）；Wuhan University（武汉大学）；DiXplorer AG（4DiXplorer公司）；Luxembourg Institute of Science and Technology (LIST)（卢森堡科学技术研究院）
Università degli Studi di Bari（巴里大学）；Politecnico di Bari（巴里理工大学）；Planetek Italia（意大利Planetek公司）
The interest for change detection in the field of remote sensing has increased in the last few years. Searching for changes in satellite images has many useful applications, ranging from land cover and land use analysis to anomaly detection. In particular, urban change detection provides an efficient tool to study urban spread and growth through several years of observation. At the same time, change detection is often a computationally challenging and time-consuming task, which requires innovative methods to guarantee optimal results with unquestionable value and within reasonable time. In thi…
University of Science and Technology of China（中国科学技术大学）
Ground Penetrating Radar (GPR) has been widely used to estimate the healthy operation of some urban roads and underground facilities. When identifying subsurface anomalies by GPR in an area, the obtained data could be unbalanced, and the numbers and types of possible underground anomalies could not be acknowledged in advance. In this paper, a novel method is proposed to improve the subsurface anomaly detection from GPR B-scan images. A normal (i.e. without subsurface objects) GPR image section is firstly collected in the detected area. Concerning that the GPR image is essentially the represent…
Denoising Diffusion Models for Anomaly Localization in Medical Images
School of Computation, Information and Technology, Technical University of Munich（慕尼黑技术大学计算、信息与技术学院）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich, Germany（生物医学影像机器学习研究所，海德堡慕尼黑德国）；Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland（巴塞尔大学生物医学工程系，瑞士Allschwil）；School of Biomedical Engineering and Imaging Sciences, King’s College London, United Kingdom（伦敦国王学院生物医学工程与影像科学学院，英国）；Department of Biomedical Informatics and Data Science, Yale University School of Medicine, New Haven, CT, USA（耶鲁大学医学院生物医学信息学与数据科学系，美国新罕布什尔州新 Haven）；Yale Biomedical Imaging Institute, Yale University, New Haven, CT, USA（耶鲁大学生物医学影像研究院，美国新罕布什尔州新 Haven）
