arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.29263cs.SD
收藏

音频幻觉攻击:测试大型音频语言模型的可靠性

Audio Hallucination Attacks: Probing the Reliability of Large Audio Language Models

University of Maryland, College Park(马里兰大学帕克分校);Adobe Research(Adobe研究院)

Ashish Seth, Sonal Kumar, Ramaneswaran Selvakumar, Nishit Anand, Utkarsh Tyagi, Prem Seetharaman, Ramani Duraiswami, Dinesh Manocha

AI总结:

本文提出Audio Hallucination Attacks,通过6500个问答对测试大型音频语言模型是否真实基于音频输入生成响应,发现其可靠性与基准性能存在差距,并提出AHA-Guard数据集降低攻击成功率。

摘要:

大型音频语言模型(LALMs)在音频-语言任务中表现优异;然而,其在现实环境中的可靠性仍待探索。我们引入Audio Hallucination Attacks(AHA),即AHA-Eval攻击套件,包含6500个问答对,用于测试LALMs是否真正基于音频输入生成响应。AHA针对两种攻击面:(i)基于查询的攻击,利用问题结构诱导关于不存在声音的幻觉;(ii)基于音频的攻击,将合成语音描述不存在的事件注入音频流。评估最先进的LALMs,包括Audio Flamingo 3和Gemini 3 Pro,我们观察到攻击成功率分别为95.35%和79.65%,揭示了被标准基准性能掩盖的可靠性差距。为缓解此问题,我们提出一个120000个问答的后对齐数据集AHA-Guard,成功将攻击成功率降低最高49%。

英文摘要:

Large Audio Language Models (LALMs) achieve strong performance on audio-language tasks; however, their reliability in real-world settings remains underexplored. We introduce Audio Hallucination Attacks (AHA), an attack suite called AHA-Eval, comprising 6.5K QA pairs designed to test whether LALMs genuinely ground their responses in the audio input. AHA targets two attack surfaces: (i) query-based attacks, which exploit question structure to induce hallucinations about absent sounds, and (ii) audio-based attacks, which inject synthetic speech describing non-existent events into the audio stream. Evaluating state-of-the-art LALMs, including Audio Flamingo 3 and Gemini 3 Pro, we observe high attack success rates of 95.35% and 79.65%, respectively, revealing a reliability gap that is hidden by standard benchmark performance. To mitigate this, we propose a 120K QA post-alignment dataset, AHA-Guard, which successfully reduces attack success rates by up to 49%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑