未见但非未知:利用数据集隐藏来稳健评估语音质量估计模型
Unseen but not Unknown: Using Dataset Concealment to Robustly Evaluate Speech Quality Estimation Models
AI总结:
通过数据集隐藏方法,研究者利用多数据集训练提升语音质量估计模型的鲁棒性与泛化能力。
摘要:
我们引入了数据集隐藏(DSC),一种新的严格程序,用于评估和解释客观语音质量估计模型。DSC量化并分解研究结果与实际应用需求之间的性能差距,同时提供有关模型行为和数据集特征的上下文和额外见解。我们还展示了通过使用对齐网络中的数据集对齐器来处理语料库效应的好处,当使用多个数据集训练模型时。我们使用九个训练数据集和九个未见数据集,以及三个已研究的模型:MOSNet、NISQA和基于Wav2Vec2.0的模型,展示了DSC和对齐器带来的改进。DSC提供了对模型泛化能力和限制的可解释视图,同时允许在训练时使用所有可用数据。此外,结果表明,在训练时将1000参数的数据集对齐器添加到9400万参数的Wav2Vec模型中,显著提高了模型对未见数据的语音质量估计能力。
英文摘要:
We introduce Dataset Concealment (DSC), a rigorous new procedure for evaluating and interpreting objective speech quality estimation models. DSC quantifies and decomposes the performance gap between research results and real-world application requirements, while offering context and additional insights into model behavior and dataset characteristics. We also show the benefits of addressing the corpus effect by using the dataset Aligner from AlignNet when training models with multiple datasets. We demonstrate DSC and the improvements from the Aligner using nine training datasets and nine unseen datasets with three well-studied models: MOSNet, NISQA, and a Wav2Vec2.0-based model. DSC provides interpretable views of the generalization capabilities and limitations of models, while allowing all available data to be used at training. An additional result is that adding the 1000 parameter dataset Aligner to the 94 million parameter Wav2Vec model during training does significantly improve the resulting model's ability to estimate speech quality for unseen data.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。