RescoreBERT：基于BERT的判别式语音识别重评分
RescoreBERT: Discriminative Speech Recognition Rescoring with BERT
摘要：
第二遍重评分是自动语音识别（ASR）系统中的重要组成部分，其用于通过实现格重评分或$n$ -最佳重排序来改善来自第一遍解码器的输出。虽然使用掩蔽语言模型（MLM）目标进行预训练在各种自然语言理解（NLU）任务中取得了巨大成功，但它并没有作为ASR的重新评分模型获得关注。具体而言，尚未探索在诸如最小WER（MWER）的判别性目标上训练双向模型（如BERT）。在这里，我们展示了如何训练具有MWER损失的基于BERT的重新评分模型，以将区分损失的改进纳入ASR的深度双向预训练模型的微调中。具体来说，我们提出了一种融合策略，将MLM融入到判别训练过程中，以有效地从预训练模型中提取知识。我们进一步提出了一种替代的歧视性损失。这种方法，我们称之为RescoreBERT，相对于LibriSpeech干净/其他测试集，在没有歧视性目标的BERT基线上，将WER降低了6.6%/3.4%。我们还在来自会话代理的内部数据集上评估了我们的方法，发现它在LSTM重新评分模型上减少了延迟和WER（相对减少了3%到8%）。
英文摘要：
Second-pass rescoring is an important component in automatic speech recognition (ASR) systems that is used to improve the outputs from a first-pass decoder by implementing a lattice rescoring or $n$-best re-ranking. While pretraining with a masked language model (MLM) objective has received great success in various natural language understanding (NLU) tasks, it has not gained traction as a rescoring model for ASR. Specifically, training a bidirectional model like BERT on a discriminative objective such as minimum WER (MWER) has not been explored. Here we show how to train a BERT-based rescoring model with MWER loss, to incorporate the improvements of a discriminative loss into fine-tuning of deep bidirectional pretrained models for ASR. Specifically, we propose a fusion strategy that incorporates the MLM into the discriminative training process to effectively distill knowledge from a pretrained model. We further propose an alternative discriminative loss. This approach, which we call RescoreBERT, reduces WER by 6.6%/3.4% relative on the LibriSpeech clean/other test sets over a BERT baseline without discriminative objective. We also evaluate our method on an internal dataset from a conversational agent and find that it reduces both latency and WER (by 3 to 8% relative) over an LSTM rescoring model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
