arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.01564cs.SDcs.LGeess.AS
收藏

将ECAPA-TDNN和Wav2Vec2 .0嵌入引入口吃检测

Introducing ECAPA-TDNN and Wav2Vec2.0 Embeddings to Stuttering Detection

CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);LORIA(洛林计算机科学与应用数学实验室);Praxiling(普拉西林实验室)

Shakeel Ahmad Sheikh, Md Sahidullah, Fabrice Hirsch, Slim Ouni

摘要:

由于可用数据集的大小有限,在口吃检测(SD)任务中采用高级深度学习(DL)架构具有挑战性。为此,这项工作介绍了使用预先训练的深度模型提取的语音嵌入的应用,这些模型是在针对不同任务的大量音频数据集上训练的。特别是,我们探索使用强调通道注意,传播和聚合时间延迟神经网络(ECAPA-TDNN)和Wav2Vec2.0模型分别在VoxCeleb和LibriSpeech数据集上训练获得的音频表示。提取嵌入后,我们基准与几个传统的分类器,如k-最近邻,高斯朴素贝叶斯,和神经网络,口吃检测任务。与仅在有限SEP-28 k数据集上训练的标准SD系统相比,我们在总体准确度方面相对于基线获得了16.74%的相对改善。最后,我们已经证明,结合两个嵌入和级联Wav2Vec2.0的多层可以进一步提高SD性能,分别高达1%和2.64%。

英文摘要:

The adoption of advanced deep learning (DL) architecture in stuttering detection (SD) tasks is challenging due to the limited size of the available datasets. To this end, this work introduces the application of speech embeddings extracted with pre-trained deep models trained on massive audio datasets for different tasks. In particular, we explore audio representations obtained using emphasized channel attention, propagation, and aggregation-time-delay neural network (ECAPA-TDNN) and Wav2Vec2.0 model trained on VoxCeleb and LibriSpeech datasets respectively. After extracting the embeddings, we benchmark with several traditional classifiers, such as a k-nearest neighbor, Gaussian naive Bayes, and neural network, for the stuttering detection tasks. In comparison to the standard SD system trained only on the limited SEP-28k dataset, we obtain a relative improvement of 16.74% in terms of overall accuracy over baseline. Finally, we have shown that combining two embeddings and concatenating multiple layers of Wav2Vec2.0 can further improve SD performance up to 1% and 2.64% respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑