arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.03689eess.AScs.SD
收藏

LUPET:将分层信息路径融入多语言 ASR

LUPET: Incorporating Hierarchical Information Path into Multilingual ASR

The Chinese University of Hong Kong(香港中文大学);GVoice, Tencent(腾讯歌神(GVoice))

Wei Liu, Jingyong Hou, Dong Yang, Muyong Cao, Tan Lee

AI总结:

本文提出 LUPET 分层信息路径,按 LID、声学单元、音素共享和 MoE token 识别逐级融合信息,在 Common Voice 10 种语言上提升多语言 ASR,并缓解高资源语言性能折损。

摘要:

为实现高性能多语言自动语音识别(ASR),多种语言信息与模型设计已分别证明了其有效性,包括语言标识(LID)、音素信息、特定语言处理模块以及跨语言自监督语音表示。人们期望在统一方案中协同利用这些优势,以进一步提升整体系统性能。本文提出一种名为 LUPET 的新型分层信息路径设计,该路径从浅层到深层,按顺序编码不同粒度尺度上的多方面语言与声学信息。该路径从 LID 预测开始,随后进行声学单元发现、音素共享,最后通过 mixture-of-expert 路由完成 token 识别。研究在 Common Voice 语料库的 10 种语言上开展 ASR 实验。结果表明,与基线系统相比,LUPET 具有更优越的性能。最重要的是,LUPET 有效缓解了多语言场景中高资源语言因低资源语言而出现性能折损的问题。

英文摘要:

Toward high-performance multilingual automatic speech recognition (ASR), various types of linguistic information and model design have demonstrated their effectiveness independently. They include language identity (LID), phoneme information, language-specific processing modules, and cross-lingual self-supervised speech representation. It is expected that leveraging their benefits synergistically in a unified solution would further improve the overall system performance. This paper presents a novel design of a hierarchical information path, named LUPET, which sequentially encodes, from the shallow layers to deep layers, multiple aspects of linguistic and acoustic information at diverse granularity scales. The path starts from LID prediction, followed by acoustic unit discovery, phoneme sharing, and finally token recognition routed by a mixture-of-expert. ASR experiments are carried out on 10 languages in the Common Voice corpus. The results demonstrate the superior performance of LUPET as compared to the baseline systems. Most importantly, LUPET effectively mitigates the issue of performance compromise of high-resource languages with low-resource ones in the multilingual setting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑