藏于噪声中：通过潜在声学模式触发器揭示音频大语言模型对齐中的后门
Hidden in the Noise: Unveiling Backdoors in Audio LLMs Alignment through Latent Acoustic Pattern Triggers
AI总结：
针对音频大语言模型（ALLM）的安全隐患，本文提出HIN后门攻击框架，构建AudioSafe基准，通过实验验证现有ALLM易受声学特征触发器攻击且该攻击隐蔽性强。
摘要：
随着音频大语言模型（Audio Large Language Models, ALLMs）成为语音处理领域的强力工具，其安全隐患亟需关注。现有大量研究已探索文本与视觉模态的安全问题，但音频的独特特性带来了显著挑战。本文首先探究：ALLM是否易受利用声学触发器的后门攻击？针对该问题，我们提出了Hidden in the Noise（HIN，藏于噪声）这一新型后门攻击框架，旨在利用细微的音频专属特征实施攻击。HIN对原始音频波形施加声学修改，包括调整时间动态特性、策略性注入频谱定制噪声，这些修改会引入ALLM声学特征编码器可捕获的一致模式，在音频流中嵌入鲁棒触发器。为评估ALLM对基于音频特征触发器的鲁棒性，我们构建了AudioSafe基准，覆盖9类不同风险类型。在AudioSafe及3个现有安全数据集上开展的大量实验揭示了现有ALLM的关键漏洞：（1）环境噪声、语速变化等音频特征可实现超过90%的平均攻击成功率；（2）ALLM对不同声学特征的敏感度差异显著，尤其对音量作为触发器的响应极弱；（3）投毒样本的加入仅造成损失曲线的微小波动，凸显了该攻击的隐蔽性。
英文摘要：
As Audio Large Language Models (ALLMs) emerge as powerful tools for speech processing, their safety implications demand urgent attention. While considerable research has explored textual and vision safety, audio's distinct characteristics present significant challenges. This paper first investigates: Is ALLM vulnerable to backdoor attacks exploiting acoustic triggers? In response to this issue, we introduce Hidden in the Noise (HIN), a novel backdoor attack framework designed to exploit subtle, audio-specific features. HIN applies acoustic modifications to raw audio waveforms, such as alterations to temporal dynamics and strategic injection of spectrally tailored noise. These changes introduce consistent patterns that an ALLM's acoustic feature encoder captures, embedding robust triggers within the audio stream. To evaluate ALLM robustness against audio-feature-based triggers, we develop the AudioSafe benchmark, assessing nine distinct risk types. Extensive experiments on AudioSafe and three established safety datasets reveal critical vulnerabilities in existing ALLMs: (I) audio features like environment noise and speech rate variations achieve over 90% average attack success rate. (II) ALLMs exhibit significant sensitivity differences across acoustic features, particularly showing minimal response to volume as a trigger, and (III) poisoned sample inclusion causes only marginal loss curve fluctuations, highlighting the attack's stealth.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
