arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.20184eess.AScs.CLcs.LGcs.SD
收藏

在数据稀缺情境下利用ASR驱动的Wav2Vec2探索病理语音质量评估

Exploring Pathological Speech Quality Assessment with ASR-Powered Wav2Vec2 in Data-Scarce Context

LIA, Avignon Université(阿维尼翁大学计算机科学与人工智能实验室);Aix-Marseille Univ, LPL, CNRS(艾克斯-马赛大学、言语与语言实验室、法国国家科学研究中心);IRIT, Université de Toulouse, CNRS, Toulouse INP, UT3(图卢兹大学、法国国家科学研究中心、图卢兹国立理工学院、图卢兹第三大学、图卢兹计算机科学研究所);IUC Toulouse, CHU Toulouse(图卢兹大学癌症研究所、图卢兹大学中心医院);Laboratoire de NeuroPsychoLinguistique, UR 4156, Université de Toulouse(图卢兹大学神经心理语言学实验室)

Tuan Nguyen, Corinne Fredouille, Alain Ghio, Mathieu Balaguer, Virginie Woisard

AI总结:

本文提出在数据稀缺条件下利用ASR驱动的预训练Wav2Vec2在音频层面进行病理语音质量评估,在HNC数据集上仅用95个样本即取得优于现有方法的结果,揭示了ASR与语音质量评估之间的强相关性。

摘要:

自动语音质量评估作为传统感知性临床评估的替代或支持手段,已受到越来越多的关注。然而,迄今为止大多数研究仅在诸如二分类之类的简单任务上取得良好结果,这在很大程度上是由于数据稀缺所致。为应对这一挑战,当前的工作倾向于将患者的音频文件切分为许多样本以扩充数据集。然而,这种方法存在局限性,因为它间接地将整体音频评分与各个片段关联起来。本文提出了一种新颖的方法,尽管面临数据稀缺,系统仍在音频层面而非片段层面进行学习。本文提出使用预训练的Wav2Vec2架构同时用于自监督学习(SSL)和作为语音评估中的特征提取器的ASR。在HNC数据集上开展实验,我们的ASR驱动方法与其他方法相比建立了新的基线,仅使用95个训练样本,在可懂度和严重程度评分预测上分别获得了平均MSE=0.73和MSE=1.15的结果。这表明基于ASR的Wav2Vec2模型带来了最佳结果,并可能表明ASR与语音质量评估之间存在强相关性。我们还测量了其在可变片段时长和语音内容上的能力,探索了影响其决策的因素。

英文摘要:

Automatic speech quality assessment has raised more attention as an alternative or support to traditional perceptual clinical evaluation. However, most research so far only gains good results on simple tasks such as binary classification, largely due to data scarcity. To deal with this challenge, current works tend to segment patients' audio files into many samples to augment the datasets. Nevertheless, this approach has limitations, as it indirectly relates overall audio scores to individual segments. This paper introduces a novel approach where the system learns at the audio level instead of segments despite data scarcity. This paper proposes to use the pre-trained Wav2Vec2 architecture for both SSL, and ASR as feature extractor in speech assessment. Carried out on the HNC dataset, our ASR-driven approach established a new baseline compared with other approaches, obtaining average $MSE=0.73$ and $MSE=1.15$ for the prediction of intelligibility and severity scores respectively, using only 95 training samples. It shows that the ASR based Wav2Vec2 model brings the best results and may indicate a strong correlation between ASR and speech quality assessment. We also measure its ability on variable segment durations and speech content, exploring factors influencing its decision.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑