arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.00648cs.CLcs.SDeess.AS
收藏

衡量自监督预训练中单个领域因素的影响

Measuring the Impact of Individual Domain Factors in Self-Supervised Pre-Training

The University of Edinburgh(爱丁堡大学);Meta AI

Ramon Sanabria, Wei-Ning Hsu, Alexei Baevski, Michael Auli

AI总结:

该研究通过对照实验拆解语音自监督预训练中各领域因素的作用,发现语音因素对预训练表征的语音识别性能影响显著,语法句法因素影响较弱,填补了相关研究空白。

摘要:

人类语音数据包含丰富的领域因素,例如口音、句法与语义多样性,或是声学环境。此前的研究整体探究了自动语音识别中预训练与微调阶段的领域不匹配所产生的影响,但并未拆解单个因素的贡献。本文开展了一项对照研究,以更好地理解这类因素对预训练表征在自动语音识别任务上性能的影响。为此,我们在经过修改的自然语音或合成音频上预训练模型(仅修改单个领域因素),随后测量微调后的性能。结果表明,语音领域因素在预训练过程中发挥重要作用,而语法与句法因素的重要性则低得多。据我们所知,这是首个旨在更深入理解语音自监督预训练中预训练集领域特征的研究。

英文摘要:

Human speech data comprises a rich set of domain factors such as accent, syntactic and semantic variety, or acoustic environment. Previous work explores the effect of domain mismatch in automatic speech recognition between pre-training and fine-tuning as a whole but does not dissect the contribution of individual factors. In this paper, we present a controlled study to better understand the effect of such factors on the performance of pre-trained representations on automatic speech recognition. To do so, we pre-train models either on modified natural speech or synthesized audio, with a single domain factor modified, and then measure performance after fine-tuning. Results show that phonetic domain factors play an important role during pre-training while grammatical and syntactic factors are far less important. To our knowledge, this is the first study to better understand the domain characteristics of pre-trained sets in self-supervised pre-training for speech.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑