arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.16028eess.AScs.SD
收藏

笑声出卖了你?从包含非语言片段的语音中学习鲁棒的说话人表征

Laugh Betrays You? Learning Robust Speaker Representation From Speech Containing Non-Verbal Fragments

Yuke Lin, Xiaoyi Qin, Huahua Cui, Zhenyi Zhu, Ming Li

AI总结:

本文研究包含非语言笑声片段的说话人确认问题,提出Laughter-Splicing based Network(LSN)方法,在LL和SL任务上分别实现20%和22%的相对提升,同时保持中性语音性能。

摘要:

自动说话人确认的成功表明,可以从中性语音中提取具有区分性的说话人表征。然而,作为一种非语言声音,笑声直觉上也应该携带说话人信息。因此,本文重点探索关于包含非语言笑声片段的语音的说话人确认。我们通过对VoxCeleb和部分CN-Celeb数据集运行笑声检测脚本,收集了一组包含笑声成分的片段。为了进一步过滤不可信片段,我们使用由纯笑声和中性语音预训练的二元笑声检测分类器计算了概率分数。此后,基于分数超过阈值的片段,我们在两种不同的评估场景下构建了测试任务:笑声-笑声(LL)和语音-笑声(SL)。随后提出了一种名为Laughter-Splicing based Network(LSN)的新方法,该方法能显著提升两种场景下的性能,并维持在中性语音(如VoxCeleb1测试集)上的表现。具体而言,我们的系统在笑声-笑声和语音-笑声任务上分别实现了20%和22%的相对提升。元数据和样本片段已发布在https://github.com/nevermoreLin/Laugh_LSN。

英文摘要:

The success of automatic speaker verification shows that discriminative speaker representations can be extracted from neutral speech. However, as a kind of non-verbal voice, laughter should also carry speaker information intuitively. Thus, this paper focuses on exploring speaker verification about utterances containing non-verbal laughter segments. We collect a set of clips with laughter components by conducting a laughter detection script on VoxCeleb and part of the CN-Celeb dataset. To further filter untrusted clips, probability scores are calculated by our binary laughter detection classifier, which is pre-trained by pure laughter and neutral speech. After that, based on the clips whose scores are over the threshold, we construct trials under two different evaluation scenarios: Laughter-Laughter (LL) and Speech-Laughter (SL). Then a novel method called Laughter-Splicing based Network (LSN) is proposed, which can significantly boost performance in both scenarios and maintain the performance on the neutral speech, such as the VoxCeleb1 test set. Specifically, our system achieves relative 20% and 22% improvement on Laughter-Laughter and Speech-Laughter trials, respectively. The meta-data and sample clips have been released at https://github.com/nevermoreLin/Laugh_LSN.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑