arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.11072cs.CLeess.AS
收藏

通过说话人不变聚类改进内容表示的自监督微调

Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering

MIT CSAIL(麻省理工学院计算机科学与人工智能实验室)

Heng-Jui Chang, Alexander H. Liu, James Glass

AI总结:

提出说话人不变聚类(Spin)自监督学习方法,通过聚类与交换预测解耦说话人信息并保留内容表示,仅需单 GPU 微调 45 分钟,在语音识别和声学单元发现任务上超越先前方法。

摘要:

自监督语音表示模型在多种任务中取得了成功,但利用无标签数据针对内容相关问题改进这些模型仍具挑战。我们提出了说话人不变聚类(Spin),这是一种新颖的自监督学习方法,它对语音表示进行聚类,并在原始与说话人扰动话语之间执行交换预测。Spin 解开了说话人信息并保留了内容表示,仅需在单个 GPU 上进行 45 分钟的微调。Spin 改进了预训练网络,并在语音识别和声学单元发现任务中超越了先前的方法。

英文摘要:

Self-supervised speech representation models have succeeded in various tasks, but improving them for content-related problems using unlabeled data is challenging. We propose speaker-invariant clustering (Spin), a novel self-supervised learning method that clusters speech representations and performs swapped prediction between the original and speaker-perturbed utterances. Spin disentangles speaker information and preserves content representations with just 45 minutes of fine-tuning on a single GPU. Spin improves pre-trained networks and outperforms prior methods in speech recognition and acoustic unit discovery.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑