Recently, researchers have shown an increasing interest in automatically predicting the subjective evaluation for speech synthesis systems. This prediction is a challenging task, especially on the out-of-domain test set. In this paper, we proposed a novel fusion model for MOS prediction that combines supervised and unsupervised approaches. In the supervised aspect, we developed an SSL-based predictor called LE-SSL-MOS. The LE-SSL-MOS utilizes pre-trained self-supervised learning models and further improves prediction accuracy by utilizing the opinion scores of each utterance in the listener enhancement branch. In the unsupervised aspect, two steps are contained: we fine-tuned the unit language model (ULM) using highly intelligible domain data to improve the correlation of an unsupervised metric - SpeechLMScore. Another is that we utilized ASR confidence as a new metric with the help of ensemble learning. To our knowledge, this is the first architecture that fuses supervised and unsupervised methods for MOS prediction. With these approaches, our experimental results on the VoiceMOS Challenge 2023 show that LE-SSL-MOS performs better than the baseline. Our fusion system achieved an absolute improvement of 13% over LE-SSL-MOS on the noisy and enhanced speech track. Our system ranked 1st and 2nd, respectively, in the French speech synthesis track and the challenge's noisy and enhanced speech track.
MOS-Bias: From Hidden Gender Bias to Gender-Aware Speech Quality Assessment
National Taiwan University(台湾大学);Nagoya University(名古屋大学);National Institute of Information and Communications Technology(信息通信研究机构);Academia Sinica(中央研究院)
SHEET: A Multi-purpose Open-source Speech Human Evaluation Estimation Toolkit
Nagoya University(名古屋大学);Japan National Institute of Information and Communications Technology(日本信息通信技术研究所);Graduate School of Informatics Information Technology Center(信息科学与技术中心研究生院)
Fusion of Self-supervised Learned Models for MOS Prediction
Graduate School of Informatics, Kyoto University(京都大学信息研究生院);National Institute of Information and Communications Technology (NICT)(日本信息与通信技术研究所);Kuaishou Technology(快手科技)
Contextual Semi-Supervised Learning: An Approach To Leverage Air-Surveillance and Untranscribed ATC Data in ASR Systems
Idiap Research Institute(伊迪亚普研究所);Ecole Polytechnique Federale de Lausanne (EPFL)(洛桑联邦理工学院);Brno University of Technology(布尔诺理工大学);ReplayWell(ReplayWell公司)
Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?
LTCI, Télécom Paris, Institut Polytechnique de Paris(巴黎理工学院巴黎电信学院LTCI);Hi! PARIS Engineering Team(Hi! PARIS工程团队);Capgemini(凯捷);Samsung AI Center(三星人工智能中心);University of Cambridge(剑桥大学);Mila-Quebec AI Institute(Mila-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);Concordia University(康考迪亚大学)
An Effective Strategy for Modeling Score Ordinality and Non-uniform Intervals in Automated Speaking Assessment
Department of Computer Science and Information Engineering, National Taiwan Normal University(国立台湾师范大学计算机科学与信息工程系);Department of Educational Psychology and Counseling, National Taiwan Normal University(国立台湾师范大学教育心理学与辅导系)
Predicting EEG Responses to Attended Speech via Deep Neural Networks for Speech
Eriksholm Research Centre(埃里克肖尔姆研究中心);Linköping University(林雪平大学);University of Stuttgart(斯图加特大学);Technical University of Denmark(丹麦技术大学);Oticon A/S(奥迪康公司)
On the Effectiveness of Speech Self-supervised Learning for Music
Queen Mary University of London(伦敦玛丽女王大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Carnegie Mellon University(卡内基梅隆大学);University of Sheffield(谢菲尔德大学);University of Waterloo(滑铁卢大学);University of Michigan Ann Arbor(密歇根大学安娜堡分校);Dartmouth College(达特茅斯学院);New York University Shanghai(上海纽约大学);Hong Kong University of Science and Technology(香港科技大学)
LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speech
Samsung AI Center Cambridge(三星剑桥人工智能中心);University of Cambridge(剑桥大学);Laboratoire Informatique d’Avignon(阿维尼翁计算机实验室);Avignon Université(阿维尼翁大学);Univ. Grenoble Alpes(格勒诺布尔大学);Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);Grenoble INP(格勒诺布尔理工学院);NAVER LABS Europe(韩国 Naver 欧洲实验室);ESPCI(巴黎高等物理化工学院);PSL Research University(巴黎文理研究大学)