大语言模型解码器是否公平？评估语言模型先验如何塑造语音识别中的偏见
Do LLM Decoders Listen Fairly? Benchmarking How Language Model Priors Shape Bias in Speech Recognition
AI总结：
研究评估九种模型在不同人口特征上的语音识别性能，发现LLM解码器在种族偏见上表现更公平，但Whisper在印度口音语音上出现病理学幻觉，且音频压缩对口音公平性影响更大。
摘要：
随着预训练大语言模型取代语音识别中的任务特定解码器，一个关键问题浮现：这些文本衍生的先验是否使识别在不同人口群体中更公平或更偏见？我们评估了九种模型，涵盖三种架构世代（CTC无语言模型、编码器-解码器隐式语言模型、基于LLM的显式预训练解码器），在Common Voice 24和Meta的Fair-Speech数据集上，约43,000个语音片段，五个人口轴（种族、口音、性别、年龄、母语）。在干净音频中，三个发现挑战了假设：LLM解码器不放大种族偏见（Granite-8B在种族公平性上最佳，最大/最小WER=2.28）；Whisper在印度口音语音上出现病理学幻觉，非单调插入率峰值达到9.62%；音频压缩预测口音公平性比LLM规模更有效。我们随后在12种声学退化条件下（噪声、回声、静音注入、块掩码）进行压力测试，总计216次推理运行。严重退化意外压缩了公平差距，所有群体均收敛到高WER，但静音注入放大Whisper的口音偏见达4.64倍，通过触发选择性幻觉。在掩码下，Whisper进入灾难性重复循环（51,797次插入中的86%），而显式LLM解码器产生38倍更少的插入并接近零重复；高压缩音频编码（Q-former）即使在LLM解码器中也重新引入重复病理学。这些结果表明，音频编码器设计而非LLM规模是公平和稳健语音识别的主要杠杆。
英文摘要：
As pretrained large language models replace task-specific decoders in speech recognition, a critical question arises: do their text-derived priors make recognition fairer or more biased across demographic groups? We evaluate nine models spanning three architectural generations (CTC with no language model, encoder-decoder with an implicit LM, and LLM-based with an explicit pretrained decoder) on about 43,000 utterances across five demographic axes (ethnicity, accent, gender, age, first language) using Common Voice 24 and Meta's Fair-Speech, a controlled-prompt dataset that eliminates vocabulary confounds. On clean audio, three findings challenge assumptions: LLM decoders do not amplify racial bias (Granite-8B has the best ethnicity fairness, max/min WER = 2.28); Whisper exhibits pathological hallucination on Indian-accented speech with a non-monotonic insertion-rate spike to 9.62% at large-v3; and audio compression predicts accent fairness more than LLM scale. We then stress-test these findings under 12 acoustic degradation conditions (noise, reverberation, silence injection, chunk masking) across both datasets, totaling 216 inference runs. Severe degradation paradoxically compresses fairness gaps as all groups converge to high WER, but silence injection amplifies Whisper's accent bias up to 4.64x by triggering demographic-selective hallucination. Under masking, Whisper enters catastrophic repetition loops (86% of 51,797 insertions) while explicit-LLM decoders produce 38x fewer insertions with near-zero repetition; high-compression audio encoding (Q-former) reintroduces repetition pathology even in LLM decoders. These results suggest that audio encoder design, not LLM scaling, is the primary lever for equitable and robust speech recognition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
