从英语到更多语言：面向跨语言语音识别的参数高效模型重编程
From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition
AI总结：
提出基于神经模型重编程的参数高效框架，通过辅助架构增强特征，将英语ASR模型重用于多语言识别，仅用极少参数即达更低WER与更高训练效率。
摘要：
在这项工作中，我们提出了一种基于神经模型重编程的参数高效学习框架，用于跨语言语音识别，该框架能够将训练良好的英语自动语音识别（ASR）模型重新用于识别其他语言。我们设计了不同的辅助神经架构，专注于可学习的预训练特征增强，这首次实现了在ASR上的模型重编程。具体而言，我们研究了如何选择基于conformer的RNN-Transducer的可训练组件（即编码器）作为冻结的预训练骨干网络。在七语言多语言LibriSpeech语音（MLS）任务上的实验表明，模型重编程仅需完整ASR模型原始可训练参数的4.2%（270M中的11M）至6.8%（660M中的45M），即可在不同语言上取得11.9%至8.1%平均WER的竞争性结果。此外，我们发现了使大规模预训练ASR在单语和多语言语音识别中均取得成功的不同设置。我们的方法在更低的WER和更好的训练效率方面，优于现有的ASR微调架构及其结合自监督损失（例如w2v-bert）的扩展版本。
英文摘要：
In this work, we propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition, which can \textbf{re-purpose} well-trained English automatic speech recognition (ASR) models to recognize the other languages. We design different auxiliary neural architectures focusing on learnable pre-trained feature enhancement that, for the first time, empowers model reprogramming on ASR. Specifically, we investigate how to select trainable components (i.e., encoder) of a conformer-based RNN-Transducer, as a frozen pre-trained backbone. Experiments on a seven-language multilingual LibriSpeech speech (MLS) task show that model reprogramming only requires 4.2% (11M out of 270M) to 6.8% (45M out of 660M) of its original trainable parameters from a full ASR model to perform competitive results in a range of 11.9% to 8.1% WER averaged across different languages. In addition, we discover different setups to make large-scale pre-trained ASR succeed in both monolingual and multilingual speech recognition. Our methods outperform existing ASR tuning architectures and their extension with self-supervised losses (e.g., w2v-bert) in terms of lower WER and better training efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
