arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.13004cs.CL

语言障碍：评估CNN与Transformer架构在语音质量估计中的跨语言性能

Language Barriers: Evaluating Cross-Lingual Performance of CNN and Transformer Architectures for Speech Quality Estimation

Technische Universität Berlin（柏林工业大学）；Deutsches Forschungszentrum für Künstliche Intelligenz（德国人工智能研究中心）

Wafaa Wardah, Tuğçe Melike Koçak Büyüktaş, Kirill Shchegelskiy, Sebastian Möller, Robert P. Spang

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文评估仅用英语训练的NISQA与AST在五种语言上的语音质量预测表现，发现AST更稳定但二者均有偏差，并指出多语言数据与架构适配的必要性。

摘要：

客观语音质量模型旨在利用自动化方法预测人类感知的语音质量。然而，跨语言泛化仍是一项重大挑战，因为平均意见得分（MOS）会因语言、感知及数据集特定差异而在不同语言之间发生变化。主要在英语数据上训练的模型可能难以泛化到具有不同语音、声调和韵律特征的语言，从而导致客观评估不一致。本研究考察了两种语音质量模型的跨语言性能：基于CNN的NISQA模型，以及基于Transformer的Audio Spectrogram Transformer（AST）模型。两种模型均仅在包含超过49,000条语音样本的英语数据集上训练，随后在德语、法语、汉语普通话、瑞典语和荷兰语语音上进行评估。我们使用Pearson相关系数（PCC）和均方根误差（RMSE），在音色、不连续性、响度、噪声和MOS五个语音质量维度上分析模型性能。研究结果表明，尽管AST取得了更稳定的跨语言性能，但两种模型都表现出明显偏差。值得注意的是，汉语普通话语音质量预测与人类MOS评分相关性很高，而瑞典语和荷兰语带来了更大的预测挑战。不连续性在所有语言中仍然难以建模。这些结果凸显出，需要更均衡的多语言数据集以及针对架构的适配，以提升跨语言泛化能力。

英文摘要：

Objective speech quality models aim to predict human-perceived speech quality using automated methods. However, cross-lingual generalization remains a major challenge, as Mean Opinion Scores (MOS) vary across languages due to linguistic, perceptual, and dataset-specific differences. A model trained primarily on English data may struggle to generalize to languages with different phonetic, tonal, and prosodic characteristics, leading to inconsistencies in objective assessments. This study investigates the cross-lingual performance of two speech quality models: NISQA, a CNN-based model, and a Transformer-based Audio Spectrogram Transformer (AST) model. Both models were trained exclusively on English datasets containing over 49,000 speech samples and subsequently evaluated on speech in German, French, Mandarin, Swedish, and Dutch. We analyze model performance using Pearson Correlation Coefficient (PCC) and Root Mean Square Error (RMSE) across five speech quality dimensions: coloration, discontinuity, loudness, noise, and MOS. Our findings show that while AST achieves a more stable cross-lingual performance, both models exhibit noticeable biases. Notably, Mandarin speech quality predictions correlate highly with human MOS scores, whereas Swedish and Dutch present greater prediction challenges. Discontinuities remain difficult to model across all languages. These results highlight the need for more balanced multilingual datasets and architecture-specific adaptations to improve cross-lingual generalization.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑