通过注意力图在推理时间检测语音大语言模型中的幻觉
Detecting Hallucinations in SpeechLLMs at Inference Time Using Attention Maps
AI总结：
本文提出四种基于注意力的指标，用于检测语音大语言模型中的幻觉，通过轻量级逻辑回归分类器实现高效推理检测，在多个任务中优于基线方法，展示了注意力模式在检测幻觉中的价值。
摘要：
语音大语言模型（SpeechLLMs）中的幻觉问题带来显著风险，但现有检测方法通常依赖于成本高昂或难以获取的黄金标准输出。此外，为文本模型开发的幻觉检测方法无法直接捕捉音频特定信号。我们研究了四个基于注意力的指标：AUDIORATIO、AUDIOCONSISTENCY、AUDIOENTROPY 和 TEXTENTROPY，旨在捕捉与幻觉相关的病理注意力模式，并在这些特征上训练轻量级逻辑回归分类器以实现高效的推理时间检测。在自动语音识别和语音到文本翻译任务中，评估显示，在Qwen-2-Audio和Voxtral-3B上，我们的方法在域内数据上优于不确定性基于和先前注意力基于的基线方法，实现了高达+0.23 PR-AUC的改进，并能泛化到域外ASR设置。我们进一步发现，使用约100个注意力头即可实现强性能，相比使用全部头提高了域外泛化能力。虽然效果依赖于模型和任务特定训练，但我们的结果表明，注意力模式为SpeechLLMs中的幻觉检测提供了一个有价值的工具。
英文摘要：
Hallucinations in Speech Large Language Models (SpeechLLMs) pose significant risks, yet existing detection methods typically rely on gold-standard outputs that are costly or impractical to obtain. Moreover, hallucination detection methods developed for text-based LLMs do not directly capture audio-specific signals. We investigate four attention-derived metrics: AUDIORATIO, AUDIOCONSISTENCY, AUDIOENTROPY, and TEXTENTROPY, designed to capture pathological attention patterns associated with hallucination, and train lightweight logistic regression classifiers on these features for efficient inference-time detection. Across automatic speech recognition and speech-to-text translation tasks, evaluations on Qwen-2-Audio and Voxtral-3B show that our approach outperforms uncertainty-based and prior attention-based baselines on in-domain data, achieving improvements of up to +0.23 PR-AUC, and generalises to out-of-domain ASR settings. We further find that strong performance can be achieved with approximately 100 attention heads, improving out-of-domain generalisation compared to using all heads. While effectiveness is model-dependent and task-specific training is required, our results demonstrate that attention patterns provide a valuable tool for hallucination detection in SpeechLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
