无监督语言不可知WER标准化
Unsupervised Language agnostic WER Standardization
摘要：
词出错率（WER）是用于评估自动语音识别（ASR）系统的标准度量。然而，WER未能提供一个公平的评价人类感知质量的拼写变化，缩写，或复合词所产生的凝集。基于区域/地理、替代缩写、借用词以及从外语到目标语言脚本的代码混合词的音译，多种拼写变体可能是可接受的。类似地，在凝集的情况下，通常凝集的以及分裂的形式是可接受的。以前的工作通过使用手动识别的归一化对来处理这个问题，并在计算WER之前将它们应用于转录和假设。在本文中，我们提出了一个自动WER规范化系统，包括两个模块：拼写规范化和分割规范化。所提出的系统是无监督和语言不可知的，因此可扩展。ASR在四种语言的35 K话语上的实验产生了13.28%的平均WER减少。这些自动识别的标准化对的人的判断表明，我们的WER标准化的评价是高度一致的ASR输出的感知质量。
英文摘要：
Word error rate (WER) is a standard metric for the evaluation of Automated Speech Recognition (ASR) systems. However, WER fails to provide a fair evaluation of human perceived quality in presence of spelling variations, abbreviations, or compound words arising out of agglutination. Multiple spelling variations might be acceptable based on locale/geography, alternative abbreviations, borrowed words, and transliteration of code-mixed words from a foreign language to the target language script. Similarly, in case of agglutination, often times the agglutinated, as well as the split forms, are acceptable. Previous work handled this problem by using manually identified normalization pairs and applying them to both the transcription and the hypothesis before computing WER. In this paper, we propose an automatic WER normalization system consisting of two modules: spelling normalization and segmentation normalization. The proposed system is unsupervised and language agnostic, and therefore scalable. Experiments with ASR on 35K utterances across four languages yielded an average WER reduction of 13.28%. Human judgements of these automatically identified normalization pairs show that our WER-normalized evaluation is highly consistent with the perceived quality of ASR output.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
