arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.00236eess.AScs.CLcs.LGcs.SD
收藏

TRILLsson:蒸馏通用副语言语音表征

TRILLsson: Distilled Universal Paralinguistic Speech Representations

Verily Life Sciences(Verily生命科学公司);Google Research(谷歌研究院)

Joel Shor, Subhashini Venugopalan

摘要:

自我监督的最新进展显着提高了语音表示的质量。然而,在设备上部署最先进的嵌入模型由于其有限的公共可用性和巨大的资源占用而受到限制。我们的工作通过公开发布一系列小型且接近最先进性能的非语言学语音模型来解决这些问题。我们的方法基于知识蒸馏,我们的模型仅基于公共数据。我们探索不同的架构,并在非语义语音(NOSS)基准上彻底评估我们的模型。我们最大的蒸馏模型的大小不到原始模型的15%(314MB vs 2.2GB),在7个任务中的6个任务上达到了96%以上的准确率,并在6.5%的数据上进行了训练。最小的模型大小为1%(22MB),在7个任务中的6个任务上达到了90%以上的准确率。我们的模型在7个任务中的6个任务上优于开源Wav2Vec 2.0模型,我们最小的模型在两个情感识别任务上都优于开源Wav2Vec 2.0,尽管大小只有开源Wav2Vec 2.0的7%。

英文摘要:

Recent advances in self-supervision have dramatically improved the quality of speech representations. However, deployment of state-of-the-art embedding models on devices has been restricted due to their limited public availability and large resource footprint. Our work addresses these issues by publicly releasing a collection of paralinguistic speech models that are small and near state-of-the-art performance. Our approach is based on knowledge distillation, and our models are distilled on public data only. We explore different architectures and thoroughly evaluate our models on the Non-Semantic Speech (NOSS) benchmark. Our largest distilled model is less than 15% the size of the original model (314MB vs 2.2GB), achieves over 96% the accuracy on 6 of 7 tasks, and is trained on 6.5% the data. The smallest model is 1% in size (22MB) and achieves over 90% the accuracy on 6 of 7 tasks. Our models outperform the open source Wav2Vec 2.0 model on 6 of 7 tasks, and our smallest model outperforms the open source Wav2Vec 2.0 on both emotion recognition tasks despite being 7% the size.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑