arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.04855cs.SDcs.AIcs.CLeess.AS
收藏

融合自监督学习模型的MOS预测

Fusion of Self-supervised Learned Models for MOS Prediction

Graduate School of Informatics, Kyoto University(京都大学信息研究生院);National Institute of Information and Communications Technology (NICT)(日本信息与通信技术研究所);Kuaishou Technology(快手科技)

Zhengdong Yang, Wangjin Zhou, Chenhui Chu, Sheng Li, Raj Dabre, Raphael Rubino, Yi Zhao

摘要:

我们参加了2022年的平均意见评分(MOS)预测挑战。这个挑战的目的是预测两个轨道上的合成语音的MOS分数,主轨道和一个更具挑战性的子轨道:域外(OOD)。为了提高预测分数的准确性,我们探索了几种模型融合相关的策略,并提出了一个融合框架,其中七个预训练的自监督学习(SSL)模型已经参与。这些预训练的SSL模型来自三个ASR框架,包括Wav2Vec,Hubert和WavLM。对于OOD轨道,我们遵循主轨道上选择的7个SSL模型,并采用半监督学习方法来利用未标记的数据。根据官方分析结果,我们的系统在16个指标中的6个指标中排名第一,并且在16个指标中的13个指标中排名前三。具体来说,我们在主音轨上的系统级LCC、SRCC和KTAU得分最高,在OOD音轨上的话语级LCC、SRCC和KTAU评估指标上的表现最好。与基本SSL模型相比,融合系统的预测精度有了很大的提高,特别是在OOD子航迹上。

英文摘要:

We participated in the mean opinion score (MOS) prediction challenge, 2022. This challenge aims to predict MOS scores of synthetic speech on two tracks, the main track and a more challenging sub-track: out-of-domain (OOD). To improve the accuracy of the predicted scores, we have explored several model fusion-related strategies and proposed a fused framework in which seven pretrained self-supervised learned (SSL) models have been engaged. These pretrained SSL models are derived from three ASR frameworks, including Wav2Vec, Hubert, and WavLM. For the OOD track, we followed the 7 SSL models selected on the main track and adopted a semi-supervised learning method to exploit the unlabeled data. According to the official analysis results, our system has achieved 1st rank in 6 out of 16 metrics and is one of the top 3 systems for 13 out of 16 metrics. Specifically, we have achieved the highest LCC, SRCC, and KTAU scores at the system level on main track, as well as the best performance on the LCC, SRCC, and KTAU evaluation metrics at the utterance level on OOD track. Compared with the basic SSL models, the prediction accuracy of the fused system has been largely improved, especially on OOD sub-track.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑