arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.08393eess.AScs.SD
收藏

SCDNet:基于自监督学习特征的说话人切换检测

SCDNet: Self-supervised Learning Feature-based Speaker Change Detection

Yue Li, Xinsheng Wang, Li Zhang, Lei Xie

AI总结:

本文提出 SCDNet 说话人切换检测模型,系统比较 Hubert、wav2vec 2.0 与 WavLm 的自监督特征,并通过可学习层加权、微调和对比学习提升性能。

摘要:

说话人切换检测(Speaker Change Detection,SCD)旨在识别对话中不同说话人之间的边界。受微调 wav2vec 2.0 模型在 SCD 任务上取得成功的启发,本文进一步研究了用于 SCD 的自监督学习(SSL)特征。具体而言,提出了一个名为 SCDNet 的 SCD 模型。借助该模型,研究了多种最先进的 SSL 模型,包括 Hubert、wav2vec 2.0 和 WavLm。为识别 SSL 模型中对 SCD 最有效的层,采用了一种可学习加权方法来分析中间表示的有效性。此外,还实现了一种基于微调的方法,以进一步比较 SSL 模型在 SCD 任务中的特性。进一步地,提出了一种对比学习方法,以缓解基于微调的方法和 SCDNet 在训练中的过拟合倾向。实验展示了 WavLm 在 SCD 任务中的优越性,也证明了 SCDNet 的良好设计。

英文摘要:

Speaker Change Detection (SCD) is to identify boundaries among speakers in a conversation. Motivated by the success of fine-tuning wav2vec 2.0 models for the SCD task, a further investigation of self-supervised learning (SSL) features for SCD is conducted in this work. Specifically, an SCD model, named SCDNet, is proposed. With this model, various state-of-the-art SSL models, including Hubert, wav2vec 2.0, and WavLm are investigated. To discern the most potent layer of SSL models for SCD, a learnable weighting method is employed to analyze the effectiveness of intermediate representations. Additionally, a fine-tuning-based approach is also implemented to further compare the characteristics of SSL models in the SCD task. Furthermore, a contrastive learning method is proposed to mitigate the overfitting tendencies in the training of both the fine-tuning-based method and SCDNet. Experiments showcase the superiority of WavLm in the SCD task and also demonstrate the good design of SCDNet.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑