arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.08917cs.CLcs.LG
收藏

基于大语言模型的大规模多语言浅层融合

Massively Multilingual Shallow Fusion with Large Language Models

Google LLC(谷歌有限责任公司)

Ke Hu, Tara N. Sainath, Bo Li, Nan Du, Yanping Huang, Andrew M. Dai, Yu Zhang, Rodrigo Cabrera, Zhifeng Chen, Trevor Strohman

摘要:

虽然大语言模型(LLM)在自然语言处理方面取得了令人印象深刻的进展,但如何利用它们来改进自动语音识别(ASR)仍然不清楚。在这项工作中,我们提出了训练一个单一的多语言语言模型(LM)在多种语言的浅融合。我们通过使用专家混合LLM来扩展多语言LM的限制,以覆盖多达84种语言,即,通用语言模型。当专家数量增加时,GLaM在每个解码步骤中动态地仅选择两个,以保持推理计算大致恒定。然后,我们将GLaM应用于基于最先进的端到端模型的多语言浅融合任务。在推理过程中,GLaM相比于类似计算的密集LM,将英语长尾测试集的WER相对降低了4.4%。在多语言浅融合任务中,GLaM改进了50种语言中的41种,平均相对WER减少了3.85%,最大减少了10%。与基线模型相比,GLaM在43种语言中的WER平均降低了5.53%。

英文摘要:

While large language models (LLM) have made impressive progress in natural language processing, it remains unclear how to utilize them in improving automatic speech recognition (ASR). In this work, we propose to train a single multilingual language model (LM) for shallow fusion in multiple languages. We push the limits of the multilingual LM to cover up to 84 languages by scaling up using a mixture-of-experts LLM, i.e., generalist language model (GLaM). When the number of experts increases, GLaM dynamically selects only two at each decoding step to keep the inference computation roughly constant. We then apply GLaM to a multilingual shallow fusion task based on a state-of-the-art end-to-end model. Compared to a dense LM of similar computation during inference, GLaM reduces the WER of an English long-tail test set by 4.4% relative. In a multilingual shallow fusion task, GLaM improves 41 out of 50 languages with an average relative WER reduction of 3.85%, and a maximum reduction of 10%. Compared to the baseline model, GLaM achieves an average WER reduction of 5.53% over 43 languages.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑