非语言发声中的说话人身份：条件蒸馏与专家混合方法
Speaker Identity in Non-Verbal Vocalizations: Conditional Distillation and Mixture of Experts Approach
AI总结：
针对非语言发声的说话人验证难题，提出结合冻结Data2Vec自监督特征、ECAPA-TDNN和专家混合模块的框架，通过条件蒸馏和对比损失，将等错误率从38.93%降至22.66%。
摘要：
随着表现性文本转语音和语音转换系统越来越多地生成非语言发声以增强自然度，可靠的说话人验证对于客观评估语言和非语言片段之间的身份一致性变得至关重要。然而，当前的说话人验证系统对非语言发声的泛化能力较差，且在非语言发声数据上微调会导致语音性能的灾难性遗忘。我们首次对10种非语言发声类型进行了系统研究，并提出了一种结合冻结的Data2Vec自监督特征与ECAPA-TDNN的框架，通过具有学习到的领域感知路由的专家混合模块进行增强。在语音输入上通过预训练教师模型的条件蒸馏损失保留了语音到语音的准确性，而对比损失则弥合了语音与非语言发声之间的领域差距。我们的方法将预训练基线的语音-非语言发声等错误率从38.93%降低到22.66%，并通过蒸馏将语音等错误率从13.17%提高到9.24%。
英文摘要：
As expressive text-to-speech (TTS) and voice conversion (VC) systems increasingly generate non-verbal vocalizations (NVVs) to enhance naturalness, reliable speaker verification (SV) becomes essential to objectively assess identity consistency across both verbal and non-verbal segments. Yet current SV systems generalize poorly to NVVs, and fine-tuning on NVV data causes catastrophic forgetting of speech performance. We present the first systematic study across 10 NVV types and propose a framework combining frozen Data2Vec self-supervised features with ECAPA-TDNN, enhanced by a Mixture of Experts (MoE) module with learned domain-aware routing. A conditional distillation loss on speech inputs via a pretrained teacher retains speech-to-speech accuracy, while a contrastive loss bridges the speech-NVV domain gap. Our method reduces speech-NVV EER from 38.93% to 22.66% over a pretrained baseline, and improves speech EER from 13.17% to 9.24% via distillation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
