arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.14092cs.CLeess.AS
收藏

无缝语言扩展:增强自监督模型的多语言掌握能力

Seamless Language Expansion: Enhancing Multilingual Mastery in Self-Supervised Models

Department of Systems Engineering and Engineering Management(系统工程与工程管理系);The Chinese University of Hong Kong(香港中文大学);Centre for Perceptual and Interactive Intelligence (CPII) Limited(感知与交互智能中心(CPII)有限公司)

Jing Xu, Minglin Wu, Xixin Wu, Helen Meng

AI总结:

本文提出将 LoRA 集成到已有自监督模型并结合数据组合与重新聚类保留策略,使 mHuBERT 高效扩展至普通话,在语音重合成任务中显著提升 MOS 并降低 WER,同时保持原有语言能力不退化。

摘要:

自监督(SSL)模型在各种下游任务中表现出色,但它们通常仅针对有限语言开发,在现实世界中可能会遇到新语言。为每种新语言开发一个 SSL 模型成本高昂,因此关键问题是如何在不损害原有能力的前提下,高效地将已有 SSL 模型适配到新语言。我们提出了将 LoRA 集成到已有 SSL 模型中的适配方法,以扩展对新语言的支持。我们还开发了保留策略,包括数据组合和重新聚类,以保持模型在已有语言上的能力。以 mHuBERT 为应用对象,我们在语音重合成任务上研究了这些方法的有效性。实验表明,我们的适配方法使 mHuBERT 能够应用于新语言(普通话),MOS 值提升约 1.6,WER 相对值最多降低 61.72%。同时,我们的保留策略确保模型在已有语言和新语言上的性能均保持完整。

英文摘要:

Self-supervised (SSL) models have shown great performance in various downstream tasks. However, they are typically developed for limited languages, and may encounter new languages in real-world. Developing a SSL model for each new language is costly. Thus, it is vital to figure out how to efficiently adapt existed SSL models to a new language without impairing its original abilities. We propose adaptation methods which integrate LoRA to existed SSL models to extend new language. We also develop preservation strategies which include data combination and re-clustering to retain abilities on existed languages. Applied to mHuBERT, we investigate their effectiveness on speech re-synthesis task. Experiments show that our adaptation methods enable mHuBERT to be applied to a new language (Mandarin) with MOS value increased about 1.6 and the relative value of WER reduced up to 61.72%. Also, our preservation strategies ensure that the performance on both existed and new languages remains intact.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑