为多语言ASR中的语码转换调整适配器
Adapting the adapters for code-switching in multilingual ASR
AI总结：
本研究提出通过整合双语言适配器信息并建模潜在二进制序列来微调多语言ASR模型，显著提升阿拉伯语、普通话和印地语与英语混合的语码转换识别性能，CER绝对降低至少10%。
摘要：
最近，大型预训练多语言语音模型在将自动语音识别（ASR）扩展到许多低资源语言方面显示出潜力。其中一些模型在其结构中采用语言适配器，这有助于提高单语性能，并避免多语言建模在资源丰富语言上的一些缺点。然而，这种结构限制了这些模型在语码转换语音上的可用性，语码转换语音是指同一话语中混合了两种语言。在这项工作中，我们提出了有效微调这些模型以处理语码转换语音的方法，通过在网络的每个语言适配点整合来自两个语言适配器的信息。我们还将语码转换建模为一系列潜在二进制序列，这些序列可用于在帧级别指导每个语言适配器的信息流。所提出的方法在三个语码转换数据集上进行了评估，这些数据集涵盖阿拉伯语、普通话和印地语与英语的配对，在所有测试集上显示出语码转换性能的一致改进，CER绝对降低至少10%。
英文摘要：
Recently, large pre-trained multilingual speech models have shown potential in scaling Automatic Speech Recognition (ASR) to many low-resource languages. Some of these models employ language adapters in their formulation, which helps to improve monolingual performance and avoids some of the drawbacks of multi-lingual modeling on resource-rich languages. However, this formulation restricts the usability of these models on code-switched speech, where two languages are mixed together in the same utterance. In this work, we propose ways to effectively fine-tune such models on code-switched speech, by assimilating information from both language adapters at each language adaptation point in the network. We also model code-switching as a sequence of latent binary sequences that can be used to guide the flow of information from each language adapter at the frame level. The proposed approaches are evaluated on three code-switched datasets encompassing Arabic, Mandarin, and Hindi languages paired with English, showing consistent improvements in code-switching performance with at least 10\% absolute reduction in CER across all test sets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
