用于语音识别中束搜索重评分的韩语Tokenization
Korean Tokenization for Beam Search Rescoring in Speech Recognition
摘要：
利用外部语言模型进行波束搜索解码可以大大提高自动语音识别模型的性能。韩国语语音识别的研究越来越受到人们的关注，但对解码过程的研究还不多。在本文中，我们提出了一个朝鲜语标记化方法的神经网络为基础的LM用于朝鲜语ASR。虽然常见的方法是使用相同的标记化方法为外部LM的ASR模型，我们表明，它可能不是最好的选择韩国。我们提出了一种新的标记化方法，插入一个特殊的令牌，SkipTC，当没有尾随辅音在韩国音节。通过利用所提出的SkipTC令牌，LM的输入序列变得非常规则，使得LM可以更好地学习语言特征。我们的实验表明，该方法实现了较低的字出错率相比，相同的LM模型没有SkipTC。此外，我们是第一个报告ASR性能为最近推出的大规模7,600小时韩国语音数据集。
英文摘要：
The performance of automatic speech recognition (ASR) models can be greatly improved by proper beam-search decoding with external language model (LM). There has been an increasing interest in Korean speech recognition, but not many studies have been focused on the decoding procedure. In this paper, we propose a Korean tokenization method for neural network-based LM used for Korean ASR. Although the common approach is to use the same tokenization method for external LM as the ASR model, we show that it may not be the best choice for Korean. We propose a new tokenization method that inserts a special token, SkipTC, when there is no trailing consonant in a Korean syllable. By utilizing the proposed SkipTC token, the input sequence for LM becomes very regularly patterned so that the LM can better learn the linguistic characteristics. Our experiments show that the proposed approach achieves a lower word error rate compared to the same LM model without SkipTC. In addition, we are the first to report the ASR performance for the recently introduced large-scale 7,600h Korean speech dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
