arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.01496cs.CLcs.LGcs.SDeess.AS
收藏

语音基础模型的有效领域自适应

Efficient Domain Adaptation for Speech Foundation Models

Google LLC(谷歌有限责任公司)

Bo Li, Dongseong Hwang, Zhouyuan Huo, Junwen Bai, Guru Prakash, Tara N. Sainath, Khe Chai Sim, Yu Zhang, Wei Han, Trevor Strohman, Francoise Beaufays

摘要:

基础模型(FM)是在广泛的数据上进行大规模训练的,并且能够适应广泛的下游任务,已经引起了研究界的极大兴趣。基础模型得益于不同模态、语言和应用领域等多种数据源,表现出较强的泛化能力和知识转移能力。在本文中,我们提出了一个开拓性的研究,建立一个有效的解决方案,基于FM的语音识别系统。我们采用最近开发的自监督BEST-RQ进行预训练,并提出使用JUST Hydra对源和无监督目标域数据进行联合微调。FM编码器适配器和解码器然后被微调到具有少量监督域内数据的目标域。在一个大规模的YouTube和语音搜索任务中,我们的方法被证明是数据和模型参数有效的。它仅使用21.6M监督域内数据和130.8M微调参数就达到了相同的质量,而731.1M模型则是在额外的300 M监督域内数据上从头开始训练的。

英文摘要:

Foundation models (FMs), that are trained on broad data at scale and are adaptable to a wide range of downstream tasks, have brought large interest in the research community. Benefiting from the diverse data sources such as different modalities, languages and application domains, foundation models have demonstrated strong generalization and knowledge transfer capabilities. In this paper, we present a pioneering study towards building an efficient solution for FM-based speech recognition systems. We adopt the recently developed self-supervised BEST-RQ for pretraining, and propose the joint finetuning with both source and unsupervised target domain data using JUST Hydra. The FM encoder adapter and decoder are then finetuned to the target domain with a small amount of supervised in-domain data. On a large-scale YouTube and Voice Search task, our method is shown to be both data and model parameter efficient. It achieves the same quality with only 21.6M supervised in-domain data and 130.8M finetuned parameters, compared to the 731.1M model trained from scratch on additional 300M supervised in-domain data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑