arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.00340cs.SDeess.AS
收藏

能否利用自监督语音特征移除说话人识别中的下游模型?

Can you Remove the Downstream Model for Speaker Recognition with Self-Supervised Speech Features?

Apple(苹果公司);Carnegie Mellon University(卡内基梅隆大学)

Zakaria Aldeneh, Takuya Higuchi, Jee-weon Jung, Skyler Seto, Tatiana Likhomanenko, Stephen Shum, Ahmed Hussen Abdelaziz, Shinji Watanabe, Barry-John Theobald

AI总结:

本文发现预训练自监督语音特征已包含说话人验证所需信息,提出大幅简化下游模型,在SUPERB上以参数减少97.51%实现29.93%的平均性能提升,并展现出更高的数据效率。

摘要:

自监督特征通常被用来替代说话人验证模型中的滤波器组特征。然而,这些模型最初是为以滤波器组特征作为输入而设计的,因此在自监督特征之上训练它们,隐含地假设两种特征类型需要相同数量的学习才能完成任务。在本工作中,我们观察到预训练的自监督语音特征本身已包含下游说话人验证任务所需的信息,因此我们可以在不牺牲性能的前提下简化下游模型。为此,我们重新审视了使用自监督特征的说话人验证下游模型的设计。我们表明,可以将模型简化到参数减少97.51%,同时在SUPERB上实现29.93%的平均性能提升。此外,我们还表明,与基线相比,简化后的下游模型具有更高的数据效率——仅使用60%的训练数据即可取得更好的性能。

英文摘要:

Self-supervised features are typically used in place of filter-bank features in speaker verification models. However, these models were originally designed to ingest filter-bank features as inputs, and thus, training them on top of self-supervised features assumes that both feature types require the same amount of learning for the task. In this work, we observe that pre-trained self-supervised speech features inherently include information required for downstream speaker verification task, and therefore, we can simplify the downstream model without sacrificing performance. To this end, we revisit the design of the downstream model for speaker verification using self-supervised features. We show that we can simplify the model to use 97.51% fewer parameters while achieving a 29.93% average improvement in performance on SUPERB. Consequently, we show that the simplified downstream model is more data efficient compared to baseline--it achieves better performance with only 60% of the training data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑