arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.05754cs.SDeess.AS
收藏

基于Cluster-Aware-DINO的自监督学习用于高性能鲁棒说话人验证

Self-Supervised Learning with Cluster-Aware-DINO for High-Performance Robust Speaker Verification

Shanghai Jiao Tong University(上海交通大学)

Bing Han, Zhengyang Chen, Yanmin Qian

AI总结:

本文提出基于聚类感知DINO的自监督说话人验证框架,通过动态损失门控与标签校正缓解不可靠伪标签问题,在Voxceleb上以更少迭代和更小模型取得显著EER改进,达到与全监督系统相当的性能。

摘要:

自动说话人验证任务利用深度学习方法和大规模人工标注数据集已取得巨大成就。然而,收集大量高质量标注数据用于系统构建非常困难且成本高昂。本文提出一种新颖且先进的自监督学习框架,可在不使用任何标注数据的情况下构建高性能说话人验证系统。为避免假阴性样本对的影响,我们采用无标签自蒸馏(DINO)框架作为初始模型,该模型无需利用负样本对即可训练。随后,我们为DINO引入一种聚类感知训练策略,以提高数据多样性。在迭代学习阶段,由于聚类产生大量不可靠标签,伪标签的质量对系统训练至关重要。这促使我们提出动态损失门控与标签校正(DLG-LC)方法,以缓解不可靠标签导致的性能下降。具体而言,我们利用高斯混合模型(GMM)对损失分布建模,并动态获得损失门控阈值以区分可靠标签与不可靠标签。此外,我们采用模型预测来校正不可靠标签,从而更好地利用不可靠数据而非直接丢弃。进一步地,我们将DLG-LC扩展到多模态以进一步提升性能。实验在常用的Voxceleb数据集上进行。与已知最佳的自监督说话人验证系统相比,我们提出的方法在Vox-O、Vox-E和Vox-H测试集上分别获得22.17%、27.94%和25.56%的相对等错误率(EER)改进,且迭代次数更少、模型更小、聚类方法更简单。更重要的是,新提出的系统甚至取得了与全监督系统相当的结果,但未使用任何人工标注数据。

英文摘要:

Automatic speaker verification task has made great achievements using deep learning approaches with the large-scale manually annotated dataset. However, it's very difficult and expensive to collect a large amount of well-labeled data for system building. In this paper, we propose a novel and advanced self-supervised learning framework which can construct a high performance speaker verification system without using any labeled data. To avoid the impact of false negative pairs, we adopt the self-distillation with no labels (DINO) framework as the initial model, which can be trained without exploiting negative pairs. Then, we introduce a cluster-aware training strategy for DINO to improve the diversity of data. In the iteration learning stage, due to a mass of unreliable labels from clustering, the quality of pseudo labels is important for the system training. This motivates us to propose dynamic loss-gate and label correction (DLG-LC) methods to alleviate the performance degradation caused by unreliable labels. More specifically, we model the loss distribution with GMM and obtain the loss-gate threshold dynamically to distinguish the reliable and unreliable labels. Besides, we adopt the model predictions to correct the unreliable label, for better utilizing the unreliable data rather than dropping them directly. Moreover, we extend the DLG-LC to multi-modality to further improve the performance. The experiments are performed on the commonly used Voxceleb dataset. Compared to the best-known self-supervised speaker verification system, our proposed method obtain 22.17%, 27.94% and 25.56% relative EER improvement on Vox-O, Vox-E and Vox-H test sets, even with fewer iterations, smaller models, and simpler clustering methods. More importantly, the newly proposed system even achieves comparable results with the fully supervised system, but without using any human labeled data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑