arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.03482eess.AScs.SD
收藏

基于监督增强声学单元的改进语音预训练

Improved Speech Pre-Training with Supervision-Enhanced Acoustic Unit

Shanghai AI Laboratory(上海人工智能实验室);iFLYTEK Research, iFLYTEK Co. Ltd.(科大讯飞股份有限公司讯飞研究院);University of Science and Technology of China(中国科学技术大学)

Pengcheng Li, Genshun Wan, Fenglin Ding, Hang Chen, Jianqing Gao, Jia Pan, Cong Liu

摘要:

语音预训练在从大规模未标记数据中学习有用和一般的潜在表示方面取得了巨大的成功。基于设计良好的自监督学习模式,预训练的模型可以用于许多下游语音任务,如自动语音识别。为了在低资源任务中充分利用标记数据,提出了一种改进的预训练方法,通过引入监督增强声学单元(SEAU)模式来强化上下文信息的表达,降低训练代价。从SEAU模式中提取的编码器表示用于生成用于HuBERT预训练过程的更具代表性的目标单元。在土库曼语语音识别任务中,分别使用500小时和100小时的微调数据,与标准HuBERT相比,所提出的方法SeHuBERT实现了10.5%和4.9%的相对字错误率降低。扩展到更多的语言和更多的数据,SeHuBERT也可以实现约10%的相对字错误率减少一半的培训成本相比,HuBERT。

英文摘要:

Speech pre-training has shown great success in learning useful and general latent representations from large-scale unlabeled data. Based on a well-designed self-supervised learning pattern, pre-trained models can be used to serve lots of downstream speech tasks such as automatic speech recognition. In order to take full advantage of the labed data in low resource task, we present an improved pre-training method by introducing a supervision-enhanced acoustic unit (SEAU) pattern to intensify the expression of comtext information and ruduce the training cost. Encoder representations extracted from the SEAU pattern are used to generate more representative target units for HuBERT pre-training process. The proposed method, named SeHuBERT, achieves a relative word error rate reductions of 10.5% and 4.9% comared with the standard HuBERT on Turkmen speech recognition task with 500 hours and 100 hours fine-tuning data respectively. Extended to more languages and more data, SeHuBERT can aslo achieve a relative word error rate reductions of approximately 10% at half of the training cost compared with HuBERT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑