arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1906.01496cs.CLcs.LGstat.ML
收藏

面向低资源域的多语言神经语言模型的正则化优势

Regularization Advantages of Multilingual Neural Language Models for Low Resource Domains

Navid Rekabsaz, Nikolaos Pappas, James Henderson, Banriskhem K. Khonglah, Srikanth Madikeri

摘要:

神经语言建模(LM)在包括自动语音识别在内的多个应用中带来了显著的改进。然而,它们通常需要大量的训练数据,这对于许多领域和语言是不可用的。在这项研究中,我们提出了一个多语言神经语言模型架构,共同训练的几个低资源语言的特定领域的数据。所提出的多语言LM由编码器和解码器中的语言特定的词嵌入和一个语言特定的LSTM层以及两个跨语言共享参数的LSTM层组成。这种多语言LM模型促进了跨语言的迁移学习,在资源非常少的情况下充当额外的正则化器。我们将我们提出的多语言方法与最先进的高度正则化的神经LM相结合,并在一系列训练数据大小的四种语言的会话数据域上进行评估。与单语LM相比,当可用的训练数据量有限时,结果显示我们提出的多语言LM的显著改进,表明跨语言参数共享在非常低资源的语言建模中的优势。

英文摘要:

Neural language modeling (LM) has led to significant improvements in several applications, including Automatic Speech Recognition. However, they typically require large amounts of training data, which is not available for many domains and languages. In this study, we propose a multilingual neural language model architecture, trained jointly on the domain-specific data of several low-resource languages. The proposed multilingual LM consists of language specific word embeddings in the encoder and decoder, and one language specific LSTM layer, plus two LSTM layers with shared parameters across the languages. This multilingual LM model facilitates transfer learning across the languages, acting as an extra regularizer in very low-resource scenarios. We integrate our proposed multilingual approach with a state-of-the-art highly-regularized neural LM, and evaluate on the conversational data domain for four languages over a range of training data sizes. Compared to monolingual LMs, the results show significant improvements of our proposed multilingual LM when the amount of available training data is limited, indicating the advantages of cross-lingual parameter sharing in very low-resource language modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑