arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2009.11394eess.AScs.LGcs.SD
收藏

FluentNet:使用深度学习的端到端语音不流利检测

FluentNet: End-to-End Detection of Speech Disfluency with Deep Learning

Queen’s University(女王大学)

Tedd Kourkounakis, Amirhossein Hajavi, Ali Etemad

摘要:

强大的演讲技巧是有价值的,在工作场所和教室环境中都很受欢迎。在可能的改善,以声乐介绍,不流利和口吃,特别是仍然是一个最常见和突出的因素,有人的示范。数以百万计的人受到口吃和其他言语不流利的影响,世界上大多数人在压力条件下交流时都会经历轻微的口吃。虽然在自动语音识别和语言模型领域已经有了很多研究,但在不流利检测和识别方面缺乏足够的工作。为此,我们提出了一个端到端的深度神经网络FluentNet,能够检测多种不同的不流利类型。FluentNet由一个挤压和激励残差卷积神经网络组成,它有助于学习强光谱帧级表示,然后是一组双向长短期记忆层,有助于学习有效的时间关系。最后,FluentNet使用了一种注意力机制来关注重要的词性,以获得更好的性能。我们进行了许多不同的实验,比较和消融研究,以评估我们的模型。我们的模型通过在公开可用的UCLASS数据集上优于该领域的其他解决方案来实现最先进的结果。此外,我们提出了LibriStutter:一个基于公共LibriSpeech数据集的不流利数据集,其中包含合成的口吃。我们还在这个数据集上评估了FluentNet,显示了我们的模型与许多基准技术相比的强大性能。

英文摘要:

Strong presentation skills are valuable and sought-after in workplace and classroom environments alike. Of the possible improvements to vocal presentations, disfluencies and stutters in particular remain one of the most common and prominent factors of someone's demonstration. Millions of people are affected by stuttering and other speech disfluencies, with the majority of the world having experienced mild stutters while communicating under stressful conditions. While there has been much research in the field of automatic speech recognition and language models, there lacks the sufficient body of work when it comes to disfluency detection and recognition. To this end, we propose an end-to-end deep neural network, FluentNet, capable of detecting a number of different disfluency types. FluentNet consists of a Squeeze-and-Excitation Residual convolutional neural network which facilitate the learning of strong spectral frame-level representations, followed by a set of bidirectional long short-term memory layers that aid in learning effective temporal relationships. Lastly, FluentNet uses an attention mechanism to focus on the important parts of speech to obtain a better performance. We perform a number of different experiments, comparisons, and ablation studies to evaluate our model. Our model achieves state-of-the-art results by outperforming other solutions in the field on the publicly available UCLASS dataset. Additionally, we present LibriStutter: a disfluency dataset based on the public LibriSpeech dataset with synthesized stutters. We also evaluate FluentNet on this dataset, showing the strong performance of our model versus a number of benchmark techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑