Current research in adversarial robustness of LLMs focuses on discrete input manipulations in the natural language space, which can be directly transferred to closed-source models. However, this approach neglects the steady progression of open-source models. As open-source models advance in capability, ensuring their safety also becomes increasingly imperative. Yet, attacks tailored to open-source LLMs that exploit full model access remain largely unexplored. We address this research gap and propose the embedding space attack, which directly attacks the continuous embedding representation of input tokens. We find that embedding space attacks circumvent model alignments and trigger harmful behaviors more efficiently than discrete attacks or model fine-tuning. Furthermore, we present a novel threat model in the context of unlearning and show that embedding space attacks can extract supposedly deleted information from unlearned LLMs across multiple datasets and models. Our findings highlight embedding space attacks as an important threat model in open-source LLMs. Trigger Warning: the appendix contains LLM-generated text with violence and harassment.
Unlearning's Blind Spots: Over-Unlearning and Prototypical Relearning Attack
Graduate School of Artificial Intelligence, Ulsan National Institute of Science and Technology (UNIST), Ulsan, South Korea（人工智能研究生院，乌山国立科学与技术研究所（UNIST），乌山，韩国）；Department of Industrial Engineering, UNIST, Ulsan, South Korea（工业工程系，UNIST，乌山，韩国）；Department of Electrical Engineering, UNIST, Ulsan, South Korea（电气工程系，UNIST，乌山，韩国）
Learning from Attacks: Attacking Variational Autoencoder for Improving Image Classification
fortiss（弗劳恩霍夫软件系统技术研究所）
对抗攻击通常被认为是对深度神经网络（DNN）鲁棒性的威胁。已经开发了各种防御技术来减轻对抗攻击对任务预测的潜在负面影响。本文从不同的角度分析了对抗攻击。也就是说，对抗性示例包含对预测有用的隐式信息，即，图像分类，并将针对DNN的对抗攻击视为能够促进特定学习任务的提取的抽象表示。我们提出了一个算法框架，利用DNN的数据自我表达和特定任务的预测的优势，以提高图像分类。该框架联合学习一个用于攻击变分自编码器（VAE）网络的DNN和一个用于分类的DNN，称为Attacking VAE for Improve Classification（AVIC）。实验结果表明，AVIC在标准数据集上的准确率高于干净样本训练和传统的对抗训练。
State Key Laboratory of Integrated Services Networks (ISN)（综合业务网理论及关键技术国家重点实验室）；Shaanxi Key Laboratory of Network and System Security, Xidian University（西安电子科技大学陕西省网络与系统安全重点实验室）；School of Data Science and Engineering, East China Normal University（华东师范大学数据科学与工程学院）；Ant Group（蚂蚁集团）；College of Computer and Data Science, Fuzhou University（福州大学计算机与数据科学学院）
Poisoning Attacks and Defenses to Federated Unlearning
ShanghaiTech University（上海科技大学）；Xidian University（西安电子科技大学）；North Carolina State University（北卡罗来纳州立大学）；University of North Texas（北德克萨斯大学）；University of Louisville（路易斯维尔大学）
Attacking LLMs and AI Agents: Advertisement Embedding Attacks Against Large Language Models
Department of Computer Science Texas A\&M University–Corpus Christi Corpus Christi, TX, USA；EECS Department University of Missouri Columbia, MO, USA；Department of Computer Engineering University of California–Riverside Riverside, CA, USA
When cheap gradients fail: the measurement cost of attacking quantum classifiers
Department of Electrical and Electronic Engineering, University of Melbourne（墨尔本大学电气与电子工程系）；School of Computing and Information Systems, University of Melbourne（墨尔本大学计算机与信息系统学院）
Improving Transferability of Adversarial Examples via Bayesian Attacks
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；School of Mathematics and Statistics, University of Glasgow（格拉斯哥大学数学与统计学学院）；Department of Computer Science, University of California, Davis（加州大学戴维斯分校计算机科学系）
Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLM
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
