arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.06329cs.CLeess.AS
收藏

面向多语言自动语音识别的参数高效语言扩展框架

A Parameter-efficient Language Extension Framework for Multilingual ASR

Tencent(腾讯)

Wei Liu, Jingyong Hou, Dong Yang, Muyong Cao, Tan Lee

AI总结:

本文提出参数高效语言扩展框架 PELE,将多语言语音识别持续学习分解为语言身份预测和跨语言适应,通过插入 Adapter 等轻量模块在低资源场景下有效缓解灾难性遗忘并优于持续联合学习。

摘要:

用单一多语言语音识别模型(MASR)覆盖所有语言非常困难,在现有 MASR 之上进行语言扩展是一种理想选择。本研究将 MASR 持续学习问题概率性地分解为语言身份预测(LP)和跨语言适应(XLA)两个子问题。基于此,我们提出了一种基于架构的语言扩展框架,能够从根本上解决灾难性遗忘问题,该框架被命名为 PELE。PELE 被设计为参数高效,通过增量引入附加模块来适应新语言。具体而言,我们探索了不同的参数高效微调(PEFT)模块及其变体,作为执行 XLA 的潜在候选方案。实验在 5 种新语言上进行,覆盖了范围广泛的不同低资源数据规模。表现最佳的 PEFT 候选方案能够在所有语言上取得令人满意的性能,并在五种语言中的三种上展现出优于持续联合学习设置的优越性。值得注意的是,专注于权重参数或输入特征的 PEFT 方法被揭示在性能上存在局限,与在层间插入轻量级模块(如 Adapter)相比,其扩展能力显著较差。

英文摘要:

Covering all languages with a multilingual speech recognition model (MASR) is very difficult. Performing language extension on top of an existing MASR is a desirable choice. In this study, the MASR continual learning problem is probabilistically decomposed into language identity prediction (LP) and cross-lingual adaptation (XLA) sub-problems. Based on this, we propose an architecture-based framework for language extension that can fundamentally solve catastrophic forgetting, debudded as PELE. PELE is designed to be parameter-efficient, incrementally incorporating an add-on module to adapt to a new language. Specifically, different parameter-efficient fine-tuning (PEFT) modules and their variants are explored as potential candidates to perform XLA. Experiments are carried out on 5 new languages with a wide range of low-resourced data sizes. The best-performing PEFT candidate can achieve satisfactory performance across all languages and demonstrates superiority in three of five languages over the continual joint learning setting. Notably, PEFT methods focusing on weight parameters or input features are revealed to be limited in performance, showing significantly inferior extension capabilities compared to inserting a lightweight module in between layers such as an Adapter.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑