Research of adversarial attacks is important for AI security because it shows the vulnerability of deep learning models and helps to build more robust models. Adversarial attacks on images are most widely studied, which include noise-based attacks, image editing-based attacks, and latent space-based attacks. However, the adversarial examples crafted by these methods often lack sufficient semantic information, making it challenging for humans to understand the failure modes of deep learning models under natural conditions. To address this limitation, we propose a natural language induced adversarial image attack method. The core idea is to leverage a text-to-image model to generate adversarial images given input prompts, which are maliciously constructed to lead to misclassification for a target model. To adopt commercial text-to-image models for synthesizing more natural adversarial images, we propose an adaptive genetic algorithm (GA) for optimizing discrete adversarial prompts without requiring gradients and an adaptive word space reduction method for improving query efficiency. We further used CLIP to maintain the semantic consistency of the generated images. In our experiments, we found that some high-frequency semantic information such as "foggy", "humid", "stretching", etc. can easily cause classifier errors. This adversarial semantic information exists not only in generated images but also in photos captured in the real world. We also found that some adversarial semantic information can be transferred to unknown classification tasks. Furthermore, our attack method can transfer to different text-to-image models (e.g., Midjourney, DALL-E 3, etc.) and image classifiers. Our code is available at: https://github.com/zxp555/Natural-Language-Induced-Adversarial-Images.
Adversarial Wear and Tear: Exploiting Natural Damage for Generating Physical-World Adversarial Examples
Department of Computer Science and Engineering, Kyung Hee University(计算机科学与工程系,庆熙大学);Technical University of Munich(慕尼黑技术大学);Seoul National University of Science and Technology(首尔科学理工大学);G-LAMP NEXUS Institute, Kyung Hee University(G-LAMP NEXUS研究所,庆熙大学)
Patch is Enough: Naturalistic Adversarial Patch against Vision-Language Pre-training Models
Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区);National University of Singapore(新加坡国立大学);Guangdong Testing Institute of Product Quality Supervision(广东省产品质量监督检验研究院)
Block Induced Signature Generative Adversarial Network (BISGAN): Signature Spoofing Using GANs and Their Evaluation
TUD Dresden University of Technology(德累斯顿工业大学);National University of Sciences and Technology (NUST)(国家科技大学);National Center of Artificial Intelligence (NCAI)(国家人工智能中心)
Defensive Adversarial CAPTCHA: A Semantics-Driven Framework for Natural Adversarial Example Generation
School of Computer and Information Engineering, Xiamen University of Technology(厦门理工学院计算机与信息工程学院);School of Computer Science, Engineering Research Center of Machine Learning and Industry Intelligence, Sichuan University(四川大学计算机科学学院、机器学习与工业智能工程研究中心);Department of Electrical and Electronic Engineering, University of Hong Kong(香港大学电子与电气工程系);Department of Computer and Information Science, Faculty of Science and Technology, University of Macau(澳门大学科学与技术学院计算机与信息科学系);Institute of Space Internet, Fudan University(复旦大学空间互联网研究所);Data61, CSIRO(澳大利亚联邦科学与工业研究组织Data61);School of Computer Engineering, Nanyang Technological University(南洋理工大学计算机工程学院)
Statistical Adversaries: Natural Backdoor-like Adversarial Features in Clean Vision Datasets
Neurint, LLC(Neurint有限责任公司);U.S. Army Cyber Corps(美国陆军网络部队);Northwestern State University of Louisiana(路易斯安那西北州立大学);Automata(Automata公司);AGH University of Krakow(克拉科夫AGH大学)
Detect and Defense Against Adversarial Examples in Deep Learning using Natural Scene Statistics and Adaptive Denoising
Univ. Rennes(雷恩大学);INSA Rennes(雷恩国立应用科学学院);CNRS(法国国家科学研究中心);IETR - UMR 6164(电子与电信研究所(UMR 6164));National Institute of Telecommunications and ICT, Oran(奥兰国家电信与信息通信技术研究所)
How do SGD hyperparameters in natural training affect adversarial robustness?
Chennai Mathematical Institute(金奈数学研究所);Microsoft Research(微软研究院)
学习率、批量大小和动量是SGD算法中三个重要的超参数。从Jastrzebski等人arXiv:1711.04623的工作中已知,神经网络的大批量训练产生不能很好地推广的模型。Yao et al. arXiv:1802.08241观察到大批量训练产生的模型具有较差的对抗鲁棒性。在同一篇论文中,作者训练模型与不同的批量大小和计算的特征值的损失函数的海森。他们观察到,随着批量大小的增加,海森的主导特征值变得更大。他们还表明,对抗性训练和小批量训练都会导致Hessian的主特征值下降或降低其频谱。他们将对抗训练和二阶信息相结合,提出了一种新的大批量训练算法,并获得了具有良好泛化能力的鲁棒模型。在本文中,我们经验性地观察了SGD超参数对使用未扰动样本训练的网络的准确性和对抗鲁棒性的影响。Jastrzebski等人考虑了具有固定学习率与批量大小比率的训练模型。他们观察到,比率越高,概括性越好。我们观察到,如Jastrzebski等人所提出的,以恒定的学习率与批量大小之比训练的网络,产量模型,推广良好,也有几乎恒定的对抗鲁棒性,独立于批量大小。我们观察到,动量是更有效的不同的批量大小和固定的学习率比恒定的学习率批量大小的比例为基础的SGD训练。
Detecting Adversarial Data by Probing Multiple Perturbations Using Expected Perturbation Score
South China University of Technology(华南理工大学);Pazhou Laboratory(琶洲实验室);The University of Melbourne(墨尔本大学);Beijing Institute of Technology(北京理工大学);Hong Kong Baptist University(香港浸会大学)