WERd：使用社会文本拼写变体评估方言语音识别
WERd: Using Social Text Spelling Variants for Evaluating Dialectal Speech Recognition
摘要：
我们研究的问题，评估自动语音识别（ASR）系统的目标方言语音输入。在这种情况下，一个主要的挑战是，方言的正字法通常是不标准的。从ASR评估的角度来看，这意味着对于预期输出没有明确的黄金标准，并且根据不同的人类注释者，几种可能的输出可以被认为是正确的，这使得标准词出错率（WER）不足以作为评估度量。这种情况是典型的机器翻译（MT），因此，我们借用的思想从MT评估指标，即TERp，翻译出错率的扩展是密切相关的WER。特别是，在将假设与参考进行比较的过程中，我们使用单词和短语的拼写变体，这些变体是我们以无监督的方式从Twitter中挖掘出来的。我们对埃及阿拉伯语的ASR输出进行评估的实验以及进一步的手动分析表明，得到的WERd（即，方言的WER度量是TERP的一种变体，它比WER更适合于评估方言的ASR。
英文摘要：
We study the problem of evaluating automatic speech recognition (ASR) systems that target dialectal speech input. A major challenge in this case is that the orthography of dialects is typically not standardized. From an ASR evaluation perspective, this means that there is no clear gold standard for the expected output, and several possible outputs could be considered correct according to different human annotators, which makes standard word error rate (WER) inadequate as an evaluation metric. Such a situation is typical for machine translation (MT), and thus we borrow ideas from an MT evaluation metric, namely TERp, an extension of translation error rate which is closely-related to WER. In particular, in the process of comparing a hypothesis to a reference, we make use of spelling variants for words and phrases, which we mine from Twitter in an unsupervised fashion. Our experiments with evaluating ASR output for Egyptian Arabic, and further manual analysis, show that the resulting WERd (i.e., WER for dialects) metric, a variant of TERp, is more adequate than WER for evaluating dialectal ASR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
