arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.02135eess.AS
收藏

微调数据对语音质量盲预测wav2vec 2.0模型的影响

Exploring the influence of fine-tuning data on wav2vec 2.0 model for blind speech quality prediction

Helard Becerra, Alessandro Ragano, Andrew Hines

摘要:

最近的研究表明,自监督模型可以产生准确的语音质量预测。由预训练的wav 2 vec 2.0模型生成的语音表示允许使用少量注释数据构建鲁棒的预测模型。这为在标记数据稀缺的情况下开发强大的模型提供了可能性。众所周知,微调提高了模型的性能;然而,目前还不清楚数据(例如,语言、样本量)影响性能。在本文中,我们探讨如何使用不同的语音语料库微调wav 2 vec 2.0可以影响其性能。我们采用了四个语音数据集,其中包含常见会议应用程序中发现的降级,并针对不同语言和数据大小的场景进行了微调wav 2 vec 2.0。在所有四个会议数据集加上一个包含合成语音的额外数据集上测试了微调模型,并将其与三个外部基线模型进行了比较。结果表明,微调模型能够与基线模型竞争。更大的微调数据保证了更好的性能;同时,语言的多样性有助于模型处理特定的语言。需要进一步的研究来评估其他使用多语言数据集预训练的wav 2 vec 2.0模型,并开发对语言多样性更具弹性的预测模型。

英文摘要:

Recent studies have shown how self-supervised models can produce accurate speech quality predictions. Speech representations generated by the pre-trained wav2vec 2.0 model allows constructing robust predicting models using small amounts of annotated data. This opens the possibility of developing strong models in scenarios where labelled data is scarce. It is known that fine-tuning improves the model's performance; however, it is unclear how the data (e.g., language, amount of samples) used for fine-tuning is influencing that performance. In this paper, we explore how using different speech corpus to fine-tune the wav2vec 2.0 can influence its performance. We took four speech datasets containing degradations found in common conferencing applications and fine-tuned wav2vec 2.0 targeting different languages and data size scenarios. The fine-tuned models were tested across all four conferencing datasets plus an additional dataset containing synthetic speech and they were compared against three external baseline models. Results showed that fine-tuned models were able to compete with baseline models. Larger fine-tune data guarantee better performance; meanwhile, diversity in language helped the models deal with specific languages. Further research is needed to evaluate other wav2vec 2.0 models pre-trained with multi-lingual datasets and to develop prediction models that are more resilient to language diversity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑