arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.08856eess.AScs.SD
收藏

面向语码转换语音识别的注意力引导适配方法

Attention-Guided Adaptation for Code-Switching Speech Recognition

National Yang Ming Chiao Tung University(国立阳明交通大学);Industrial Technology Research Institute(工业技术研究院)

Bobbi Aditya, Mahdin Rohmatillah, Liang-Hsuan Tai, Jen-Tzung Chien

AI总结:

本文提出注意力引导适配方法,通过筛选并引导表达语言身份的注意力头,以仅增加5.6%参数的高效方式提升Whisper在汉英语码转换语音识别上的表现,混合错误率降至14.2%。

摘要:

以Whisper为代表的强大 multilingual 模型的普及,显著推动了语音识别研究的发展。然而,这些模型在处理语码转换场景时往往表现不佳,而该场景在多语言语音识别中至关重要。近期研究尝试通过为不同语言分离模块来应对这一场景,以确保语言具有不同的潜在表示;另一些方法则考虑基于语言识别的切换机制。本研究提出了一种新的注意力引导适配方法,用于对双语 ASR 进行参数高效学习。该方法首先在模型中选取那些能够较强表达语言身份的注意力头,然后引导这些注意力头正确关注其对应的语言。在普通话-英语语码转换语音语料库上的实验表明,所提方法取得了 14.2% 的混合错误率,超越了 state-of-the-art 方法,且仅在 Whisper 基础上训练了 5.6% 的额外参数。

英文摘要:

The prevalence of the powerful multilingual models, such as Whisper, has significantly advanced the researches on speech recognition. However, these models often struggle with handling the code-switching setting, which is essential in multilingual speech recognition. Recent studies have attempted to address this setting by separating the modules for different languages to ensure distinct latent representations for languages. Some other methods considered the switching mechanism based on language identification. In this study, a new attention-guided adaptation is proposed to conduct parameter-efficient learning for bilingual ASR. This method selects those attention heads in a model which closely express language identities and then guided those heads to be correctly attended with their corresponding languages. The experiments on the Mandarin-English code-switching speech corpus show that the proposed approach achieves a 14.2% mixed error rate, surpassing state-of-the-art method, where only 5.6% additional parameters over Whisper are trained.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑