arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.07280cs.CLcs.SDeess.AS
收藏

回放以记忆:用于德语语音识别的持续层级特定微调

Replay to Remember: Continual Layer-Specific Fine-tuning for German Speech Recognition

University of Hamburg(汉堡大学)

Theresa Pekarek Rosin, Stefan Wermter

AI总结:

本文针对大规模ASR模型微调新领域时原始领域性能下降的问题,利用SVC-de数据集,通过层级特定微调和Experience Replay持续学习,在新领域达到低于5%的WER并稳定通用识别性能。

摘要:

尽管自动语音识别(ASR)模型随着无监督或自监督训练技术的引入取得了显著进展,但这些改进仍仅限于部分语言和说话人。迁移学习使得大规模多语言模型不仅能适应低资源语言,还能适应更特定的说话人群体。然而,基于新领域数据的微调通常伴随着原始领域性能的下降。因此,在我们的实验中,利用我们自己的德语老年语音命令(SVC-de)数据集,我们考察了大规模ASR模型在较小领域上的性能逼近程度,以及通过在训练期间选择性冻结部分模型能保留多少通用语音识别性能。为了进一步提高ASR模型对微调领域之外词汇和说话人的鲁棒性,我们应用了Experience Replay进行持续学习。仅通过添加原始领域的一小部分数据,我们就能在新领域上达到低于5%的词错率(WER),同时将通用语音识别的性能稳定在可接受的WER范围内。

英文摘要:

While Automatic Speech Recognition (ASR) models have shown significant advances with the introduction of unsupervised or self-supervised training techniques, these improvements are still only limited to a subsection of languages and speakers. Transfer learning enables the adaptation of large-scale multilingual models to not only low-resource languages but also to more specific speaker groups. However, fine-tuning on data from new domains is usually accompanied by a decrease in performance on the original domain. Therefore, in our experiments, we examine how well the performance of large-scale ASR models can be approximated for smaller domains, with our own dataset of German Senior Voice Commands (SVC-de), and how much of the general speech recognition performance can be preserved by selectively freezing parts of the model during training. To further increase the robustness of the ASR model to vocabulary and speakers outside of the fine-tuned domain, we apply Experience Replay for continual learning. By adding only a fraction of data from the original domain, we are able to reach Word-Error-Rates (WERs) below 5\% on the new domain, while stabilizing performance for general speech recognition at acceptable WERs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑