Most current methods generate adversarial examples with the $L_p$ norm specification. As a result, many defense methods utilize this property to eliminate the impact of such attacking algorithms. In this paper,we instead introduce "unrestricted" perturbations that create adversarial samples by using spurious relations which were learned by model training. Specifically, we find feature clusters in non-semantic features that are strongly correlated with model judgment results, and treat them as spurious relations learned by the model. Then we create adversarial samples by using them to replace the corresponding feature clusters in the target image. Experimental evaluations show that in both black-box and white-box situations. Our adversarial examples do not change the semantics of images, while still being effective at fooling an adversarially trained DNN image classifier.
Academy for Engineering and Technology, Fudan University（复旦大学工程与应用技术研究院）；Youtu Lab, Tencent（腾讯优图实验室）；School of Computer Science, Fudan University（复旦大学计算机学院）
Department of Computer Science, University of Oxford（牛津大学计算机科学系（注：这里按要求提取到大学级别，实际机构是牛津大学，其下的计算机科学系作为所属机构））
Neural networks are vulnerable to adversarially-constructed perturbations of their inputs. Most research so far has considered perturbations of a fixed magnitude under some $l_p$ norm. Although studying these attacks is valuable, there has been increasing interest in the construction of (and robustness to) unrestricted attacks, which are not constrained to a small and rather artificial subset of all possible adversarial inputs. We introduce a novel algorithm for generating such unrestricted adversarial inputs which, unlike prior work, is adaptive: it is able to tune its attacks to the classifi…
Adversarial examples are typically constructed by perturbing an existing data point within a small matrix norm, and current defense methods are focused on guarding against this type of attack. In this paper, we propose unrestricted adversarial examples, a new threat model where the attackers are not restricted to small norm-bounded perturbations. Different from perturbation-based attacks, we propose to synthesize unrestricted adversarial examples entirely from scratch using conditional generative models. Specifically, we first train an Auxiliary Classifier Generative Adversarial Network (AC-GA…
Adversarial examples are data points misclassified by neural networks. Originally, adversarial examples were limited to adding small perturbations to a given image. Recent work introduced the generalized concept of unrestricted adversarial examples, without limits on the added perturbations. In this paper, we introduce a new category of attacks that create unrestricted adversarial examples for object detection. Our key idea is to generate adversarial objects that are unrelated to the classes identified by the target object detector. Different from previous attacks, we use off-the-shelf Generat…
We introduce an approach that enhances images using a color filter in order to create adversarial effects, which fool neural networks into misclassification. Our approach, Adversarial Color Enhancement (ACE), generates unrestricted adversarial images by optimizing the color filter via gradient descent. The novelty of ACE is its incorporation of established practice for image enhancement in a transparent manner. Experimental results validate the white-box adversarial strength and black-box transferability of ACE. A range of examples demonstrates the perceptual quality of images that ACE produce…
Pohang University of Science and Technology (POSTECH)（浦项科技大学）
Adversarial examples are inputs intentionally generated for fooling a deep neural network. Recent studies have proposed unrestricted adversarial attacks that are not norm-constrained. However, the previous unrestricted attack methods still have limitations to fool real-world applications in a black-box setting. In this paper, we present a novel method for generating unrestricted adversarial examples using GAN where an attacker can only access the top-1 final decision of a classification model. Our method, Latent-HSJA, efficiently leverages the advantages of a decision-based attack in the laten…
We propose a novel approach for generating unrestricted adversarial examples by manipulating fine-grained aspects of image generation. Unlike existing unrestricted attacks that typically hand-craft geometric transformations, we learn stylistic and stochastic modifications leveraging state-of-the-art generative models. This allows us to manipulate an image in a controlled, fine-grained manner without being bounded by a norm threshold. Our approach can be used for targeted and non-targeted unrestricted attacks on classification, semantic segmentation and object detection models. Our attacks can …
ReToMe-VA: Recursive Token Merging for Video Diffusion-based Unrestricted Adversarial Attack
School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化研究院）
Thanks to recent advances in deep neural networks (DNNs), face recognition systems have become highly accurate in classifying a large number of face images. However, recent studies have found that DNNs could be vulnerable to adversarial examples, raising concerns about the robustness of such systems. Adversarial examples that are not restricted to small perturbations could be more serious since conventional certified defenses might be ineffective against them. To shed light on the vulnerability to such adversarial examples, we propose a flexible and efficient method for generating unrestricted…
Peking University（北京大学）；University of Waterloo（滑铁卢大学）
Transfer-based adversarial example is one of the most important classes of black-box attacks. However, there is a trade-off between transferability and imperceptibility of the adversarial perturbation. Prior work in this direction often requires a fixed but large $\ell_p$-norm perturbation budget to reach a good transfer success rate, leading to perceptible adversarial perturbations. On the other hand, most of the current unrestricted adversarial attacks that aim to generate semantic-preserving perturbations suffer from weaker transferability to the target model. In this work, we propose a geo…
Deep neural networks have been shown to be vulnerable to adversarial examples deliberately constructed to misclassify victim models. As most adversarial examples have restricted their perturbations to $L_{p}$-norm, existing defense methods have focused on these types of perturbations and less attention has been paid to unrestricted adversarial examples; which can create more realistic attacks, able to deceive models without affecting human predictions. To address this problem, the proposed adversarial attack generates an unrestricted adversarial example with a limited number of parameters. The…
Conventional object detection methods essentially suppose that the training and testing data are collected from a restricted target domain with expensive labeling cost. For alleviating the problem of domain dependency and cumbersome labeling, this paper proposes to detect objects in an unrestricted environment by leveraging domain knowledge trained from an auxiliary source domain with sufficient labels. Specifically, we propose a multi-adversarial Faster-RCNN (MAF) framework for unrestricted object detection, which inherently addresses domain disparity minimization for domain adaptation in fea…
The Hong Kong Polytechnic University（香港理工大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
Utilizing 3D point cloud data has become an urgent need for the deployment of artificial intelligence in many areas like facial recognition and self-driving. However, deep learning for 3D point clouds is still vulnerable to adversarial attacks, e.g., iterative attacks, point transformation attacks, and generative attacks. These attacks need to restrict perturbations of adversarial examples within a strict bound, leading to the unrealistic adversarial 3D point clouds. In this paper, we propose an Adversarial Graph-Convolutional Generative Adversarial Network (AdvGCGAN) to generate visually real…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Machine learning models, especially deep neural networks (DNNs), have been shown to be vulnerable against adversarial examples which are carefully crafted samples with a small magnitude of the perturbation. Such adversarial perturbations are usually restricted by bounding their $\mathcal{L}_p$ norm such that they are imperceptible, and thus many current defenses can exploit this property to reduce their adversarial impact. In this paper, we instead introduce "unrestricted" perturbations that manipulate semantically meaningful image-based visual descriptors - color and texture - in order to gen…
Institute of Image Communication and Network Engineering, Shanghai JiaoTong University（上海交通大学图像通信与网络工程研究所）
Traditional adversarial examples are typically generated by adding perturbation noise to the input image within a small matrix norm. In practice, un-restricted adversarial attack has raised great concern and presented a new threat to the AI safety. In this paper, we propose a wavelet-VAE structure to reconstruct an input image and generate adversarial examples by modifying the latent code. Different from perturbation-based attack, the modifications of the proposed method are not limited but imperceptible to human eyes. Experiments show that our method can generate high quality adversarial exam…
Alibaba Group（阿里巴巴集团）；Tsinghua University（清华大学）；RealAI（瑞莱智慧）；Shanghai Jiao Tong University（上海交通大学）；Peking University（北京大学）；University of Waterloo（滑铁卢大学）；Beijing Institute of Technology（北京理工大学）；Guangzhou University（广州大学）；Beihang University（北京航空航天大学）；KAIST（韩国科学技术院）；Army Engineering University of PLA（中国人民解放军陆军工程大学）
Many works have investigated the adversarial attacks or defenses under the settings where a bounded and imperceptible perturbation can be added to the input. However in the real-world, the attacker does not need to comply with this restriction. In fact, more threats to the deep model come from unrestricted adversarial examples, that is, the attacker makes large and visible modifications on the image, which causes the model classifying mistakenly, but does not affect the normal observation in human perspective. Unrestricted adversarial attack is a popular and practical direction but has not bee…
Due to the inherent robustness of segmentation models, traditional norm-bounded attack methods show limited effect on such type of models. In this paper, we focus on generating unrestricted adversarial examples for semantic segmentation models. We demonstrate a simple and effective method to generate unrestricted adversarial examples using conditional generative adversarial networks (CGAN) without any hand-crafted metric. The naïve implementation of CGAN, however, yields inferior image quality and low attack success rate. Instead, we leverage the SPADE (Spatially-adaptive denormalization) stru…
We introduce a two-player contest for evaluating the safety and robustness of machine learning systems, with a large prize pool. Unlike most prior work in ML robustness, which studies norm-constrained adversaries, we shift our focus to unconstrained adversaries. Defenders submit machine learning models, and try to achieve high accuracy and coverage on non-adversarial data while making no confident mistakes on adversarial inputs. Attackers try to subvert defenses by finding arbitrary unambiguous inputs where the model assigns an incorrect label with high confidence. We propose a simple unambigu…
Queen Mary University of London（伦敦玛丽女王大学）；Centre for Intelligent Sensing（智能传感中心）
Adversarial attacks that generate small L_p-norm perturbations to mislead classifiers have limited success in black-box settings and with unseen classifiers. These attacks are also not robust to defenses that use denoising filters and to adversarial training procedures. Instead, adversarial attacks that generate unrestricted perturbations are more robust to defenses, are generally more successful in black-box settings and are more transferable to unseen classifiers. However, unrestricted perturbations may be noticeable to humans. In this paper, we propose a content-based black-box adversarial …
Enhancing Diffusion-based Unrestricted Adversarial Attacks via Adversary Preferences Alignment
College of Intelligent Robotics and Advanced Manufacturing, Fudan University（智能机器人与先进制造学院，复旦大学）；Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University（上海智能信息处理重点实验室，计算机科学与人工智能学院，复旦大学）；Peking University（北京大学）；vivo Mobile Communication Co., Ltd（vivo移动通信有限公司）
University of Electronic Science and Technology of China（电子科技大学）
Unrestricted color attacks, which manipulate semantically meaningful color of an image, have shown their stealthiness and success in fooling both human eyes and deep neural networks. However, current works usually sacrifice the flexibility of the uncontrolled setting to ensure the naturalness of adversarial examples. As a result, the black-box attack performance of these methods is limited. To boost transferability of adversarial examples without damaging image quality, we propose a novel Natural Color Fool (NCF) which is guided by realistic color distributions sampled from a publicly availabl…
Adversarially Guided Diffusion for LiDAR Range Image Synthesis
School of Electrical and Computer Engineering, National Technical University of Athens（雅典国立技术大学电气与计算机工程学院）；Industrial Systems Institute, Athena Research Center（雅典娜研究中心工业系统研究所）
SIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy
School of Computer and Information Engineering, Xiamen University of Technology（厦门理工学院计算机与信息工程学院）；Department of Electrical and Electronic Engineering, University of Hong Kong（香港大学电子与电气工程系）；School of Computer Science, Engineering Research Center of Machine Learning and Industry Intelligence, Sichuan University（四川大学计算机科学学院，机器学习与工业智能工程研究中心）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；College of Artificial Intelligence, Southwest University（西南大学人工智能学院）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；Department of Computer and Information Science, Faculty of Science and Technology, University of Macau（澳门大学科技学院计算机与信息科学系）；Data61, CSIRO（CSIRO Data61）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Nanyang Technological University（南洋理工大学）；East China Normal University（华东师范大学）；Agency for Science, Technology and Research（新加坡科技研究局）；Meta AI；Shanghai Industrial Control Safety Innovation Tech. Co., Ltd（上海工控安全创新科技有限公司）；Zhejiang Sci-Tech University（浙江理工大学）
Transcending Adversarial Perturbations: Manifold-Aided Adversarial Examples with Legitimate Semantics
College of Information Science and Engineering, Northeastern University（东北大学信息科学与工程学院）；Foshan Graduate School, Northeastern University（东北大学佛山研究生院）；College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）
Huazhong University of Science and Technology（华中科技大学）；Peking University（北京大学）；Cornell University（康奈尔大学）
Despite the rapid development of adversarial machine learning, most adversarial attack and defense researches mainly focus on the perturbation-based adversarial examples, which is constrained by the input images. In comparison with existing works, we propose non-constrained adversarial examples, which are generated entirely from scratch without any constraint on the input. Unlike perturbation-based attacks, or the so-called unrestricted adversarial attack which is still constrained by the input noise, we aim to learn the distribution of adversarial examples to generate non-constrained but sema…
Nanjing University of Science and Technology（南京理工大学）；Peking University（北京大学）
Despite the success of deep learning across various domains, it remains vulnerable to adversarial attacks. Although many existing adversarial attack methods achieve high success rates, they typically rely on $\ell_{p}$-norm perturbation constraints, which do not align with human perceptual capabilities. Consequently, researchers have shifted their focus toward generating natural, unrestricted adversarial examples (UAEs). GAN-based approaches suffer from inherent limitations, such as poor image quality due to instability and mode collapse. Meanwhile, diffusion models have been employed for UAE …
Beijing Institute of Technology（北京理工大学）；Nanyang Normal University（南阳师范学院）
Deep neural networks (DNNs) have been found to be vulnerable to adversarial examples. Adversarial examples are malicious images with visually imperceptible perturbations. While these carefully crafted perturbations restricted with tight $\Lp$ norm bounds are small, they are still easily perceivable by humans. These perturbations also have limited success rates when attacking black-box models or models with defenses like noise reduction filters. To solve these problems, we propose Demiguise Attack, crafting ``unrestricted'' perturbations with Perceptual Similarity. Specifically, we can create p…
In this work, we propose a novel framework named Coconditional Autoencoding Adversarial Networks (CocoAAN) for Chinese font learning, which jointly learns a generation network and two encoding networks of different feature domains using an adversarial process. The encoding networks map the glyph images into style and content features respectively via the pairwise substitution optimization strategy, and the generation network maps these two kinds of features to glyph samples. Together with a discriminative network conditioned on the extracted features, our framework succeeds in producing realis…
SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；The Chinese University of Hong Kong（香港中文大学）；Uber ATG（优步先进技术集团）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Nanyang Technological University（南洋理工大学）；Alibaba Group（阿里巴巴集团）；Tianjin University（天津大学）；State Administration of Cultural Heritage（国家文物局）
Image denoising can remove natural noise that widely exists in images captured by multimedia devices due to low-quality imaging sensors, unstable image transmission processes, or low light conditions. Recent works also find that image denoising benefits the high-level vision tasks, e.g., image classification. In this work, we try to challenge this common sense and explore a totally new problem, i.e., whether the image denoising can be given the capability of fooling the state-of-the-art deep neural networks (DNNs) while enhancing the image quality. To this end, we initiate the very first attem…
Center for Research on Intelligent Perception and Computing（智能感知与计算研究中心）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；College of Automation（自动化学院）；Chinese Academy of Sciences, Institute of Automation（中国科学院自动化研究所）
With the identity information in face data more closely related to personal credit and property security, people pay increasing attention to the protection of face data privacy. In different tasks, people have various requirements for face de-identification (De-ID), so we propose a systematical solution compatible for these De-ID operations. Firstly, an attribute disentanglement and generative network is constructed to encode two parts of the face, which are the identity (facial features like mouth, nose and eyes) and expression (including expression, pose and illumination). Through face swapp…
Improving Adversarial Transferability on Vision-Language Pre-training Models via Surrogate-Specific Bias Correction
School of Cyber Science and Engineering, Southeast University（东南大学网络空间安全学院）；Purple Mountain Laboratories（紫金山实验室）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）
In recent years, Deep Learning(DL) techniques have been extensively deployed for computer vision tasks, particularly visual classification problems, where new algorithms reported to achieve or even surpass the human performance. While many recent works demonstrated that DL models are vulnerable to adversarial examples. Fortunately, generating adversarial examples usually requires white-box access to the victim model, and real-world cloud-based image classification services are more complex than white-box classifier,the architecture and parameters of DL models on cloud platforms cannot be obtai…
The collaborative reasoning for understanding each image-question pair is very critical but under-explored for an interpretable Visual Question Answering (VQA) system. Although very recent works also tried the explicit compositional processes to assemble multiple sub-tasks embedded in the questions, their models heavily rely on the annotations or hand-crafted rules to obtain valid reasoning layout, leading to either heavy labor or poor performance on composition reasoning. In this paper, to enable global context reasoning for better aligning image and language domains in diverse and unrestrict…
Black-box Attacks on Image Activity Prediction and its Natural Language Explanations
Idiap Research Institute（伊迪亚普研究所）；University of Perugia（佩鲁贾大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Queen Mary University of London（伦敦玛丽女王大学）
UC Santa Cruz（加州大学圣克鲁兹分校）；Leia Inc.（Leia公司）；NVIDIA（英伟达）
As is expressed in the adage "a picture is worth a thousand words", when using spoken language to communicate visual information, brevity can be a challenge. This work describes a novel technique for leveraging machine-learned feature embeddings to sonify visual (and other types of) information into a perceptual audio domain, allowing users to perceive this information using only their aural faculty. The system uses a pretrained image embedding network to extract visual features and embed them in a compact subset of Euclidean space -- this converts the images into feature vectors whose $L^2$ d…
