USM-SCD：基于大型预训练基础模型的多语言说话人变更检测
USM-SCD: Multilingual Speaker Change Detection Based on Large Pretrained Foundation Models
AI总结：
提出多语言说话人变更检测模型USM-SCD，基于大型语音基础模型微调，在96种语言上实现高精度检测与ASR，仅需训练四分之一参数即达SOTA性能。
摘要：
我们引入了一个多语言说话人变更检测模型（USM-SCD），该模型能够同时检测说话人轮次并对96种语言执行ASR。该模型改编自在大量监督和无监督数据上训练的语音基础模型，展示了从大型通用基础模型进行微调以用于下游任务的实用性。我们通过一系列消融研究分析了该多语言说话人变更检测模型的性能。我们表明，USM-SCD模型在由96种语言数据组成的测试集上，能够实现超过75%的平均说话人变更检测F1分数。在美国英语上，USM-SCD模型在各种公开和内部测试集上能够实现85.8%的说话人变更检测F1分数，相对之前的单语基线模型提升了21%。我们还表明，只需微调四分之一的可训练模型参数即可实现最佳的模型性能。与强大的公开ASR基线相比，USM-SCD模型展现出最先进的ASR质量，使其能够以可忽略的额外计算成本处理这两项任务。
英文摘要：
We introduce a multilingual speaker change detection model (USM-SCD) that can simultaneously detect speaker turns and perform ASR for 96 languages. This model is adapted from a speech foundation model trained on a large quantity of supervised and unsupervised data, demonstrating the utility of fine-tuning from a large generic foundation model for a downstream task. We analyze the performance of this multilingual speaker change detection model through a series of ablation studies. We show that the USM-SCD model can achieve more than 75% average speaker change detection F1 score across a test set that consists of data from 96 languages. On American English, the USM-SCD model can achieve an 85.8% speaker change detection F1 score across various public and internal test sets, beating the previous monolingual baseline model by 21% relative. We also show that we only need to fine-tune one-quarter of the trainable model parameters to achieve the best model performance. The USM-SCD model exhibits state-of-the-art ASR quality compared with a strong public ASR baseline, making it suitable to handle both tasks with negligible additional computational cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
