在转录中迷失，在分布偏移中发现：揭开语音基础模型中幻觉的面纱
Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models
AI总结：
本文研究语音基础模型中ASR幻觉问题，提出幻觉错误率HER指标，发现分布偏移与HER强相关，并揭示传统WER指标可能掩盖或隐藏幻觉风险。
摘要：
以模型规模和数据规模而言均在大规模上训练的语音基础模型，产生了能够执行多种语音任务（包括自动语音识别（ASR））的稳健系统。这些模型超越了语言和领域的障碍，但有效衡量其性能仍然是一项挑战。传统指标如词错误率（WER）和字符错误率（CER）通常用于评估ASR性能，但在关键语境中往往无法反映转录质量，尤其是在检测虚构输出时。这一被称为幻觉的现象在医疗、法律和航空等高风险领域中尤其令人担忧，因为错误可能带来严重后果。在我们的工作中，我们通过研究ASR模型中的幻觉来弥补这一空白。我们考察了分布偏移、模型规模和模型架构等因素如何影响幻觉错误率（HER），这是我们为量化幻觉而引入的一个指标。我们对超过20个ASR模型的分析揭示了若干关键见解：（1）高WER可能掩盖低幻觉率，而低WER可能隐藏危险的幻觉。（2）合成噪声，包括对抗性扰动以及白噪声、音高偏移和时间拉伸等常见扰动，都会提高HER。（3）分布偏移与HER强相关（α=0.91）。我们的发现强调了将HER与传统指标（如WER）结合使用以更好地评估ASR模型性能的重要性，尤其是在高风险领域中。
英文摘要：
Speech foundation models trained at a massive scale, both in terms of model and data size, result in robust systems capable of performing multiple speech tasks, including automatic speech recognition (ASR). These models transcend language and domain barriers, yet effectively measuring their performance remains a challenge. Traditional metrics like word error rate (WER) and character error rate (CER) are commonly used to evaluate ASR performance but often fail to reflect transcription quality in critical contexts, particularly when detecting fabricated outputs. This phenomenon, known as hallucination, is especially concerning in high-stakes domains such as healthcare, legal, and aviation, where errors can have severe consequences. In our work, we address this gap by investigating hallucination in ASR models. We examine how factors such as distribution shifts, model size, and model architecture influence the hallucination error rate (HER), a metric we introduce to quantify hallucinations. Our analysis of over 20 ASR models reveals \numinsights~key insights: (1) High WERs can mask low hallucination rates, while low WERs may conceal dangerous hallucinations. (2) Synthetic noise, both adversarial and common perturbations like white noise, pitch shift, and time stretching, increase HER. (3) Distribution shift correlates strongly with HER ($α= 0.91$). Our findings highlight the importance of incorporating HER alongside traditional metrics like WER to better assess ASR model performance, particularly in high-stakes domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
