As large language models (LLMs) become increasingly integrated into daily life, audio has emerged as a key interface for human-AI interaction. However, this convenience also introduces new vulnerabilities, making audio a potential attack surface for adversaries. Our research introduces WhisperInject, a two-stage adversarial audio attack framework that manipulates state-of-the-art audio language models to generate harmful content. Our method embeds harmful payloads as subtle perturbations into audio inputs that remain intelligible to human listeners. The first stage uses a novel reward-based white-box optimization method, Reinforcement Learning with Projected Gradient Descent (RL-PGD), to jailbreak the target model and elicit harmful native responses. This native harmful response then serves as the target for Stage 2, Payload Injection, where we use gradient-based optimization to embed subtle perturbations into benign audio carriers, such as weather queries or greeting messages. Our method achieves average attack success rates of 60-78% across two benchmarks and five multimodal LLMs, validated by multiple evaluation frameworks. Our work demonstrates a new class of practical, audio-native threats, moving beyond theoretical exploits to reveal a feasible and covert method for manipulating multimodal AI systems.
from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);People’s Public Security University of China(中国人民公安大学);Tsinghua University(清华大学)
Geneshift: Impact of different scenario shift on Jailbreaking LLM
National University of Singapore(新加坡国立大学);NUS Graduate School(新加坡国立大学研究生院);Integrative Sciences and Engineering Programme(整合科学与工程项目);Institute of Data Science (IDS)(数据科学研究院);School of Computing(计算机学院)
OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing
Institute of Software, Chinese Academy of Sciences, Beijing, China;Science \& Technology on Integrated Information System Laboratory, Beijing, China;State Key Laboratory of Complex System Modeling;University of Chinese Academy of Sciences, Beijing, China
Activation-Guided Local Editing for Jailbreaking Attacks
Beihang University(北航);Hangzhou Innovation Institute(杭州创新研究院);HKUST(香港科技大学);South China University of Technology(华南理工大学);Nanyang Technological University(南洋理工大学);Kunming University of Science and Technology(昆明理工大学)
Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking
School of Computer Science and Engineering, Sun Yat-Sen University(中山大学计算机科学与工程学院);Nanyang Technological University(南洋理工大学);Guangdong Province Key Laboratory of Information Security Technology(广东省信息安全技术重点实验室)
Jailbreaking on Text-to-Video Models via Scene Splitting Strategy
Yonsei University(延世大学);Korea Institute of Science and Technology(韩国科学技术院);AIM Intelligence(AIM智能);Seoul National University(首尔国立大学);Kyung Hee University(庆熙大学)
Speechmatics(Speechmatics公司);MATS(机器学习对齐理论与实验项目(注:MATS全称为Machine Learning Alignment Theory and Experiments,为独立研究机构,此处保留缩写并补充说明));UCL(伦敦大学学院);Stanford University(斯坦福大学);University of Oxford(牛津大学);Tangentic(Tangentic公司);Anthropic(Anthropic公司)
ALMGuard: Safety Shortcuts and Where to Find Them as Guardrails for Audio-Language Models
Beijing University of Posts and Telecommunications(北京邮电大学);National University of Singapore(新加坡国立大学);CSIRO’s Data61(CSIRO数据61);Responsible AI Research (RAIR) Centre, The University of Adelaide(负责任人工智能研究(RAIR)中心,阿德莱德大学);Tsinghua University(清华大学)
Acoustic Interference: A New Paradigm Weaponizing Acoustic Latent Semantic for Universal Jailbreak against Large Audio Language Models
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));The Hong Kong Polytechnic University(香港理工大学);The Chinese University of Hong Kong(香港中文大学)
A Survey of Large Audio Language Models: Generalization, Trustworthiness, and Outlook
Nanyang Technological University(南洋理工大学);Independent Researcher(独立研究者);The University of Melbourne(墨尔本大学);North China Electric Power University(华北电力大学);Beijing University of Posts and Telecommunications(北京邮电大学);University of Chinese Academy of Sciences(中国科学院大学);University of Science and Technology of China(中国科学技术大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Shanghai AI Laboratory(上海人工智能实验室);Huazhong University of Science and Technology(华中科技大学);Tsinghua University(清华大学);Fortemedia Singapore(富媒体新加坡);Tencent(腾讯);Fudan University(复旦大学);Wuhan University(武汉大学);Chinese University of Hong Kong(香港中文大学);Chongqing University of Posts and Telecommunications(重庆邮电大学);University of Illinois Chicago(伊利诺伊大学芝加哥分校)