具有语言信息的非洲语言多语言语音识别评估
Linguistically Informed Evaluation of Multilingual ASR for African Languages
AI总结：
本文通过引入FER和TER，评估了多语言语音识别模型在非洲语言中的表现，发现声调特征是主要挑战，FER能更准确揭示模型错误模式。
摘要：
词错误率（WER）通过将语音学、声调和其他语言错误合并为一个词汇错误，错误地表征了非洲语言的语音识别模型性能。相比之下，特征错误率（FER）最近引起了关注，作为一种揭示模型性能中语言上有意义的错误的可行度量标准。在本文中，我们通过结合 CER、FER 和一个声调意识扩展（TER）来评估三种语音编码器在两种非洲语言上的表现。我们显示，通过在语音学特征上计算错误，FER 和 TER 即使在词级准确性仍然较低时，也能揭示出语言上有意义的错误模式。我们的结果表明，模型在段落特征上表现更好，而声调（尤其是中声和下步声调）仍然是最具有挑战性的特征。在约鲁巴语上的结果显示出指标之间的显著差异，WER=0.788，CER=0.305，FER=0.151。同样，对于乌内梅语（一种濒危语言，未出现在预训练数据中），一个模型具有接近全错的WER和0.461 CER，却达到了相对较低的FER 0.267。这表明模型的错误往往归因于个别语音特征错误，而像WER这样的全或无度量标准会掩盖这一点。
英文摘要：
Word Error Rate (WER) mischaracterizes ASR models' performance for African languages by combining phonological, tone, and other linguistic errors into a single lexical error. By contrast, Feature Error Rate (FER) has recently attracted attention as a viable metric that reveals linguistically meaningful errors in models' performance. In this paper, we evaluate three speech encoders on two African languages by complementing WER with CER, and FER, and add a tone-aware extension (TER). We show that by computing errors on phonological features, FER and TER reveal linguistically-salient error patterns even when word-level accuracy remains low. Our results reveal that models perform better on segmental features, while tones (especially mid and downstep) remain the most challenging features. Results on Yoruba show a striking differential in metrics, with WER=0.788, CER=0.305, and FER=0.151. Similarly for Uneme (an endangered language absent from pretraining data) a model with near-total WER and 0.461 CER achieves the relatively low FER of 0.267. This indicates model error is often attributable to individual phonetic feature errors, which is obscured by all-or-nothing metrics like WER.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
