In this paper, we tackle the detection of out-of-distribution (OOD) objects in semantic segmentation. By analyzing the literature, we found that current methods are either accurate or fast but not both which limits their usability in real world applications. To get the best of both aspects, we propose to mitigate the common shortcomings by following four design principles: decoupling the OOD detection from the segmentation task, observing the entire segmentation network instead of just its output, generating training data for the OOD detector by leveraging blind spots in the segmentation network and focusing the generated data on localized regions in the image to simulate OOD objects. Our main contribution is a new OOD detection architecture called ObsNet associated with a dedicated training scheme based on Local Adversarial Attacks (LAA). We validate the soundness of our approach across numerous ablation studies. We also show it obtains top performances both in speed and accuracy when compared to ten recent methods of the literature on three different datasets.
SAM Encoder Breach by Adversarial Simplicial Complex Triggers Downstream Model Failures
State Key Laboratory of Advanced Rail Autonomous Operation（先进轨道交通自主运行状态关键实验室）；Beijing Key Lab of Traffic Data Mining and Embodied Intelligence（北京交通数据挖掘与具身智能重点实验室）；Collaborative Innovation Center of Railway Traffic Safety（铁路交通安全协同创新中心）；National Engineering Research Center of Rail Transportation Operation and Control System（铁路运输操作与控制系统国家工程研究中心）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；School of Automation and Intelligence, Beijing Jiaotong University（北京交通大学自动化与智能学院）
University of Cagliari（卡利亚里大学）；Pluribus One；University of Genoa（热那亚大学）；Google（谷歌）；CINI（意大利国家信息学大学联盟）
Evaluating robustness of machine-learning models to adversarial examples is a challenging problem. Many defenses have been shown to provide a false sense of robustness by causing gradient-based attacks to fail, and they have been broken under more rigorous evaluations. Although guidelines and best practices have been suggested to improve current adversarial robustness evaluations, the lack of automatic testing and debugging tools makes it difficult to apply these recommendations in a systematic manner. In this work, we overcome these limitations by: (i) categorizing attack failures based on ho…
Failure Cases Are Better Learned But Boundary Says Sorry: Facilitating Smooth Perception Change for Accuracy-Robustness Trade-Off in Adversarial Training
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））
AdvSerial: Physical Adversarial Attacks on Infrastructure-mounted Pedestrian Detectors via Semantic Feature Suppression
School of Transportation Science and Engineering, Beihang University（北京航空航天大学交通科学与工程学院）；State Key Lab of Intelligent Transportation System（智能交通系统国家重点实验室）；Zhongguancun Laboratory（中关村实验室）
Text-Guided Diffusion-Based Adversarial Attacks on Chest X-Ray Images
The Johns Hopkins University（约翰斯·霍普金斯大学）；Manipal Institute of Technology（马尼帕尔理工学院）；Manipal Academy of Higher Education（马尼帕尔高等教育学院）；The Johns Hopkins Hospital（约翰斯·霍普金斯医院）；Columbia University Irving Medical Center（哥伦比亚大学欧文医学中心）
Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models
Mohamed Bin Zayed University of AI, UAE（穆罕默德·本·扎耶德人工智能大学，阿联酋）；Khalifa University, UAE（卡比拉大学，阿联酋）；Australian National University, Australia（澳大利亚国立大学，澳大利亚）
Adversarial Training has proved to be an effective training paradigm to enforce robustness against adversarial examples in modern neural network architectures. Despite many efforts, explanations of the foundational principles underpinning the effectiveness of Adversarial Training are limited and far from being widely accepted by the Deep Learning community. In this paper, we describe surprising properties of adversarially-trained models, shedding light on mechanisms through which robustness against adversarial attacks is implemented. Moreover, we highlight limitations and failure modes affecti…
Vector Institute（向量研究所）；University of Toronto（多伦多大学）；University of Bremen（不来梅大学）；Center for Industrial Mathematics（工业数学中心）；University of Tübingen（图宾根大学）
Despite their impressive performance, deep neural networks exhibit striking failures on out-of-distribution inputs. One core idea of adversarial example research is to reveal neural network errors under such distribution shifts. We decompose these errors into two complementary sources: sensitivity and invariance. We show deep networks are not only too sensitive to task-irrelevant changes of their input, as is well-known from epsilon-adversarial examples, but are also too invariant to a wide range of task-relevant changes, thus making vast regions in input space vulnerable to adversarial attack…
Statistical Adversaries: Natural Backdoor-like Adversarial Features in Clean Vision Datasets
Neurint, LLC（Neurint有限责任公司）；U.S. Army Cyber Corps（美国陆军网络部队）；Northwestern State University of Louisiana（路易斯安那西北州立大学）；Automata（Automata公司）；AGH University of Krakow（克拉科夫AGH大学）
Model-agnostic Adversarial Attack and Defense for Vision-Language-Action Models
The University of Auckland（奥克兰大学）；King Abdullah University of Science and Technology（国王 Abdullah 科学与技术大学）；Tokyo University of Science（东京科学大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）
Anhui University（安徽大学）；University of Science and Technology of China（中国科学技术大学）
In order to prevent illegal or unauthorized access of image data such as human faces and ensure legitimate users can use authorization-protected data, reversible adversarial attack technique is rise. Reversible adversarial examples (RAE) get both attack capability and reversibility at the same time. However, the existing technique can not meet application requirements because of serious distortion and failure of image recovery when adversarial perturbations get strong. In this paper, we take advantage of Reversible Image Transformation technique to generate RAE and achieve reversible adversari…
Triggering hallucinations in model-based MRI reconstruction via adversarial perturbations
Faculty of Engineering and Architecture/Computer Engineering（工程与建筑学院/计算机工程学院）；Department of Computer Science, Mathematics and Statistics（计算机科学、数学与统计学系）；Ghent University（根特大学）；Data Mining and Modeling for Biomedicine Group（生物医学数据挖掘与建模小组）；VIB Inflammation Research Center（VIB炎症研究中心）
Technische Universität Darmstadt（达姆施塔特工业大学）；Fraunhofer IGD（弗劳恩霍夫计算机图形学研究所）
Failure cases of black-box deep learning, e.g. adversarial examples, might have severe consequences in healthcare. Yet such failures are mostly studied in the context of real-world images with calibrated attacks. To demystify the adversarial examples, rigorous studies need to be designed. Unfortunately, complexity of the medical images hinders such study design directly from the medical images. We hypothesize that adversarial examples might result from the incorrect mapping of image space to the low dimensional generation manifold by deep networks. To test the hypothesis, we simplify a complex…
Prognostics and Health Management (PHM) is an emerging engineering discipline which is concerned with the analysis and prediction of equipment health and performance. One of the key challenges in PHM is to accurately predict impending failures in the equipment. In recent years, solutions for failure prediction have evolved from building complex physical models to the use of machine learning algorithms that leverage the data generated by the equipment. However, failure prediction problems pose a set of unique challenges that make direct application of traditional classification and prediction a…
University of California, Merced（加州大学默塞德分校）；Beijing University of Chemical Technology（北京化工大学）
Failure detection is employed in the industry to improve system performance and reduce costs due to unexpected malfunction events. So, a good dataset of the system is desirable for designing an automated failure detection system. However, industrial process datasets are unbalanced and contain little information about failure behavior due to the uniqueness of these events and the high cost for running the system just to get information about the undesired behaviors. For this reason, performing correct training and validation of automated failure detection methods is challenging. This paper prop…
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；School of Transportation Science and Engineering, Beihang University（北京航空航天大学交通科学与工程学院）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；SenseTime Research（商汤科技研究院）；China University of Petroleum (East China)（中国石油大学（华东））；Peng Cheng Laboratory（鹏城实验室）；Case Western Reserve University（凯斯西储大学）
Although weakly-supervised techniques can reduce the labeling effort, it is unclear whether a saliency model trained with weakly-supervised data (e.g., point annotation) can achieve the equivalent performance of its fully-supervised version. This paper attempts to answer this unexplored question by proving a hypothesis: there is a point-labeled dataset where saliency models trained on it can achieve equivalent performance when trained on the densely annotated dataset. To prove this conjecture, we proposed a novel yet effective adversarial trajectory-ensemble active learning (ATAL). Our contrib…
Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Models
Beijing University of Posts and Telecommunications（北京邮电大学）；Jiangnan University（江南大学）；Nanyang Technological University（南洋理工大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
Neural network based classifiers are still prone to manipulation through adversarial perturbations. State of the art attacks can overcome most of the defense or detection mechanisms suggested so far, and adversaries have the upper hand in this arms race. Adversarial examples are designed to resemble the normal input from which they were constructed, while triggering an incorrect classification. This basic design goal leads to a characteristic spatial behavior within the context of Activation Spaces, a term coined by the authors to refer to the hyperspaces formed by the activation values of the…
Curvature Under Attack in hZACH-ViT: Gauge Symmetry, Boundary Saturation, and Adversarial Failure
BioML, Research Institute CODE, University of the Bundeswehr Munich（慕尼黑联邦国防军大学 CODE 研究所 BioML）；Amsterdam UMC, University of Amsterdam（阿姆斯特丹大学阿姆斯特丹医学中心）
