从文本度量到模型内部：Whisper ASR幻觉检测研究
From Text Metrics to Model Internals: A Study of Whisper ASR Hallucination Detection
AI总结：
研究Whisper大模型在真实语音数据上的幻觉检测，通过文本、LLM和内部解码器状态三种方法，发现无参考文本时探测解码器表示性能最强，融合文本与内部状态的元分类器效果最佳。
摘要：
ASR模型的幻觉——流畅但无音频基础的转录——会降低系统性能并在下游应用中带来风险。对此类错误的稳健检测仍然是一个挑战。本文研究了Whisper large v3在真实语音人工标注数据上的幻觉检测，涵盖三种范式：基于文本的、基于LLM的和内部解码器状态探测。利用文本评估度量的文本分类器实现了高召回率，但在没有参考转录时性能下降。基于LLM的检测通过领域特定的提示条件提高了精确率，但仍不如轻量级的文本方法。在没有真实参考的情况下探测Whisper的解码器表示取得了最强性能，揭示了幻觉特征在中间解码层中被编码。结合文本和内部状态输出的后期融合元分类器实现了最佳的整体检测性能。
英文摘要：
Hallucinations of ASR models - fluent transcriptions with no basis in audio - degrade system performance and pose risks in downstream applications. Robust detection of such errors remains a challenge. This paper studies Whisper large v3 hallucination detection on real-speech human-annotated data across three paradigms: text-based, LLM-based, and internal decoder state probing. Text classifiers utilizing metrics for text evaluation achieve high recall but degrade without reference transcripts. LLM-based detection improves precision with domain-specific prompt conditioning, yet remains less competitive than the lightweight text-based methods. Probing Whisper's decoder representations, without a ground-truth reference, yields the strongest performance, revealing that hallucination traits are encoded across intermediate decoding layers. A late-fusion meta-classifier combining text and internal-state outputs achieves the best overall detection performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
