arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.15903eess.AScs.SDeess.SP
收藏

具有两步多模态深度净化的说话人识别

Speaker recognition with two-step multi-modal deep cleansing

National University of Singapore(新加坡国立大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))

Ruijie Tao, Kong Aik Lee, Zhan Shi, Haizhou Li

摘要:

基于神经网络的说话人识别近年来取得了显著的进展。鲁棒的说话人表示从训练集中的硬样本和简单样本中学习有意义的知识,以实现良好的性能。然而,噪声样本(即,具有错误标签)会引起混淆,并导致网络学习不正确的表示。在本文中,我们提出了一个两步的视听深度净化框架,以消除说话人表示学习中的噪声标签的影响。这个框架包含一个粗粒度的清洗步骤来搜索特殊的样本,然后是一个细粒度的清洗步骤来过滤掉嘈杂的标签。我们的研究从一个高效的视听说话人识别系统开始,该系统在Vox-O、E和H测试集上分别达到了接近完美的等误率(EER)0.01%、0.07%和0.13%。使用所提出的多模态净化机制,四个不同的说话人识别网络实现了平均5.9%的改进。代码已在以下网址提供:\textcolor{magenta}{\url{https://github.com/TaoRuijie/AVCleanse}}。

英文摘要:

Neural network-based speaker recognition has achieved significant improvement in recent years. A robust speaker representation learns meaningful knowledge from both hard and easy samples in the training set to achieve good performance. However, noisy samples (i.e., with wrong labels) in the training set induce confusion and cause the network to learn the incorrect representation. In this paper, we propose a two-step audio-visual deep cleansing framework to eliminate the effect of noisy labels in speaker representation learning. This framework contains a coarse-grained cleansing step to search for the peculiar samples, followed by a fine-grained cleansing step to filter out the noisy labels. Our study starts from an efficient audio-visual speaker recognition system, which achieves a close to perfect equal-error-rate (EER) of 0.01\%, 0.07\% and 0.13\% on the Vox-O, E and H test sets. With the proposed multi-modal cleansing mechanism, four different speaker recognition networks achieve an average improvement of 5.9\%. Code has been made available at: \textcolor{magenta}{\url{https://github.com/TaoRuijie/AVCleanse}}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑