arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.09885eess.AScs.AI
收藏

一种改进的用于自动语音识别的单步非自回归Transformer

An Improved Single Step Non-autoregressive Transformer for Automatic Speech Recognition

University of California, Los Angeles(加利福尼亚大学洛杉矶分校);PAII Inc.(PAII公司)

Ruchao Fan, Wei Chu, Peng Chang, Jing Xiao, Abeer Alwan

摘要:

非自回归机制可以显着减少语音Transformers的推理时间,特别是当单步变量被应用时。基于CTC的单步非自回归Transformer(CASS-NAT)的先前工作已经显示出比自回归Transformers(AT)大的实时因子(RTF)改进。在这项工作中,我们提出了几种方法来提高端到端CASS-NAT的准确性,其次是性能分析。首先,卷积增强的自注意块被应用于编码器和解码器模块。其次,我们建议扩大每个令牌的触发掩模(声学边界),以增加CTC对齐的鲁棒性。此外,迭代损失函数用于增强低层参数的梯度更新。在不使用外部语言模型的情况下,改进后的CASS-NAT在Librispeech测试clean/other集上的WER分别为3.1%/7.2%,在Aishell 1测试集上的CER为5.4%,WER/CER相对提高了7%~21%。为了进行分析,我们绘制了解码器中的注意力权重分布,以可视化标记级声学嵌入之间的关系。当声学嵌入可视化时,我们发现它们具有与单词嵌入相似的行为,这解释了为什么改进的CASS-NAT的性能与AT相似。

英文摘要:

Non-autoregressive mechanisms can significantly decrease inference time for speech transformers, especially when the single step variant is applied. Previous work on CTC alignment-based single step non-autoregressive transformer (CASS-NAT) has shown a large real time factor (RTF) improvement over autoregressive transformers (AT). In this work, we propose several methods to improve the accuracy of the end-to-end CASS-NAT, followed by performance analyses. First, convolution augmented self-attention blocks are applied to both the encoder and decoder modules. Second, we propose to expand the trigger mask (acoustic boundary) for each token to increase the robustness of CTC alignments. In addition, iterated loss functions are used to enhance the gradient update of low-layer parameters. Without using an external language model, the WERs of the improved CASS-NAT, when using the three methods, are 3.1%/7.2% on Librispeech test clean/other sets and the CER is 5.4% on the Aishell1 test set, achieving a 7%~21% relative WER/CER improvement. For the analyses, we plot attention weight distributions in the decoders to visualize the relationships between token-level acoustic embeddings. When the acoustic embeddings are visualized, we find that they have a similar behavior to word embeddings, which explains why the improved CASS-NAT performs similarly to AT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑