arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.04296cs.CLcs.LG
收藏

面向语音处理的 BEST-RQ 开放实现与研究

Open Implementation and Study of BEST-RQ for Speech Processing

Laboratoire Informatique d'Avignon(阿维尼翁信息学实验室);Avignon Université(阿维尼翁大学);Samsung AI Center(三星人工智能中心);Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);Grenoble INP(格勒诺布尔理工学院);LIG(信息与信号处理实验室)

Ryan Whetten, Titouan Parcollet, Marco Dinarelli, Yannick Estève

AI总结:

本文重新实现并开放了 BEST-RQ 中的随机投影量化器,在四个下游任务上与 wav2vec 2.0 对比,证明其能以超过两倍的训练效率达到相似性能。

摘要:

自监督学习(SSL)已被证明在多种语音任务中有效。然而,这些方法通常在数据、内存和计算资源方面要求很高。基于 BERT 的随机投影量化器语音预训练(BEST-RQ)是一种 SSL 方法,在自动语音识别(ASR)上表现出色,同时比其他 SSL 方法(如 wav2vec 2.0)更简单。尽管 BEST-RQ 性能优异,但原始论文缺少细节,例如预训练所使用的 GPU/TPU 小时数,并且没有官方易用的开源实现。此外,除 ASR 和语音翻译外,BEST-RQ 尚未在其他下游任务上进行评估。在本工作中,我们描述了对随机投影量化器的重新实现,并在四个下游任务上与 wav2vec 2.0 进行了初步比较研究。我们讨论了实现细节和差异。我们表明,随机投影量化器可以达到与 wav2vec 2.0 相似的下游性能,同时将训练时间缩短超过两倍。

英文摘要:

Self-Supervised Learning (SSL) has proven to be useful in various speech tasks. However, these methods are generally very demanding in terms of data, memory, and computational resources. BERT-based Speech pre-Training with Random-projection Quantizer (BEST-RQ), is an SSL method that has shown great performance on Automatic Speech Recognition (ASR) while being simpler than other SSL methods, such as wav2vec 2.0. Despite BEST-RQ's great performance, details are lacking in the original paper, such as the amount of GPU/TPU hours used in pre-training, and there is no official easy-to-use open-source implementation. Furthermore, BEST-RQ has not been evaluated on other downstream tasks aside from ASR and speech translation. In this work, we describe a re-implementation of a Random-projection quantizer and perform a preliminary study with a comparison to wav2vec 2.0 on four downstream tasks. We discuss the details and differences of our implementation. We show that a random projection quantizer can achieve similar downstream performance as wav2vec 2.0 while decreasing training time by over a factor of two.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑