arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.10656eess.AS
收藏

LE-SSL-MOS:带听者增强的自监督学习MOS预测

LE-SSL-MOS: Self-Supervised Learning MOS Prediction with Listener Enhancement

Zili Qi, Xinhui Hu, Wangjin Zhou, Sheng Li, Hao Wu, Jian Lu, Xinkang Xu

AI总结:

本文提出LE-SSL-MOS融合模型,结合监督与无监督方法预测语音合成MOS,在VoiceMOS Challenge 2023上优于基线,并在法语和噪声增强轨道上排名前列。

摘要:

近年来,研究者们对自动预测语音合成系统主观评价的兴趣日益增加。这一预测任务具有挑战性,尤其是在域外测试集上。本文提出了一种新颖的融合模型用于MOS预测,该模型结合了监督和无监督方法。在监督方面,我们开发了一个基于SSL的预测器,称为LE-SSL-MOS。LE-SSL-MOS利用预训练的自监督学习模型,并通过在听者增强分支中利用每个话语的意见分数进一步提高预测准确性。在无监督方面,包含两个步骤:我们使用高可懂度的领域数据微调单元语言模型(ULM),以提高无监督指标SpeechLMScore的相关性;另一个是借助集成学习,我们利用ASR置信度作为新指标。据我们所知,这是首个融合监督和无监督方法进行MOS预测的架构。通过这些方法,我们在VoiceMOS Challenge 2023上的实验结果表明,LE-SSL-MOS优于基线。我们的融合系统在噪声和增强语音轨道上比LE-SSL-MOS实现了13%的绝对提升。我们的系统分别在法语语音合成轨道和挑战赛的噪声与增强语音轨道中排名第一和第二。

英文摘要:

Recently, researchers have shown an increasing interest in automatically predicting the subjective evaluation for speech synthesis systems. This prediction is a challenging task, especially on the out-of-domain test set. In this paper, we proposed a novel fusion model for MOS prediction that combines supervised and unsupervised approaches. In the supervised aspect, we developed an SSL-based predictor called LE-SSL-MOS. The LE-SSL-MOS utilizes pre-trained self-supervised learning models and further improves prediction accuracy by utilizing the opinion scores of each utterance in the listener enhancement branch. In the unsupervised aspect, two steps are contained: we fine-tuned the unit language model (ULM) using highly intelligible domain data to improve the correlation of an unsupervised metric - SpeechLMScore. Another is that we utilized ASR confidence as a new metric with the help of ensemble learning. To our knowledge, this is the first architecture that fuses supervised and unsupervised methods for MOS prediction. With these approaches, our experimental results on the VoiceMOS Challenge 2023 show that LE-SSL-MOS performs better than the baseline. Our fusion system achieved an absolute improvement of 13% over LE-SSL-MOS on the noisy and enhanced speech track. Our system ranked 1st and 2nd, respectively, in the French speech synthesis track and the challenge's noisy and enhanced speech track.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑