arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.05966cs.SDcs.CLcs.LGeess.AS
收藏

面向自监督语音模型性能的早期预测

Towards Early Prediction of Self-Supervised Speech Model Performance

Avignon UniversitéFrance(阿维尼翁大学);Thales SIXFrance(泰勒斯集团);Samsung AI Center CambridgeUnited Kingdom(三星人工智能中心);Université Grenoble AlpesFrance(格勒诺布尔阿尔卑斯大学);Laboratoire Informatique d’Avignon(阿维尼翁计算机实验室);Laboratoire d’Informatique de Grenoble(格勒诺布尔计算机实验室)

Ryan Whetten, Lucas Maison, Titouan Parcollet, Marco Dinarelli, Yannick Estève

AI总结:

本文提出通过测量SSL语音模型嵌入的聚类质量和秩,在仅用一小时无标签音频的条件下实现比预训练损失更准确的下游性能早期预测,从而降低评估成本。

摘要:

在自监督学习(SSL)中,预训练和评估都是资源密集型的。在语音领域,当前用于衡量预训练期间SSL模型质量的指标(例如损失)与下游性能的相关性并不好。因此,在预训练期间通常难以以低成本的方式判断最终的下游性能。在这项工作中,我们提出了无监督的高效方法,用于洞察SSL语音模型预训练的质量,即测量SSL模型嵌入的聚类质量和秩。结果表明,仅使用一小时的无标签音频,聚类质量和秩的度量与下游性能的相关性优于预训练损失,从而减少了SSL模型评估中对GPU小时数和标注数据的需求。

英文摘要:

In Self-Supervised Learning (SSL), pre-training and evaluation are resource intensive. In the speech domain, current indicators of the quality of SSL models during pre-training, such as the loss, do not correlate well with downstream performance. Consequently, it is often difficult to gauge the final downstream performance in a cost efficient manner during pre-training. In this work, we propose unsupervised efficient methods that give insights into the quality of the pre-training of SSL speech models, namely, measuring the cluster quality and rank of the embeddings of the SSL model. Results show that measures of cluster quality and rank correlate better with downstream performance than the pre-training loss with only one hour of unlabeled audio, reducing the need for GPU hours and labeled data in SSL model evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑