结合Conformer CPC和Deep Cluster的语音表示学习,参加2021年ZeroSpeech挑战赛
Speech Representation Learning Combining Conformer CPC with Deep Cluster for the ZeroSpeech Challenge 2021
摘要:
我们为2021年零资源语音挑战赛提出了一个系统,该系统将对比预测编码(CPC)与深度聚类相结合。在深度聚类中,我们首先准备通过使用k-means对CPC网络的输出进行聚类而获得的伪标签。然后,我们训练一个额外的自回归模型,以监督的方式对先前获得的伪标签进行分类。音素判别表示是通过执行第二轮聚类与自回归模型的最后一层的输出。我们表明,替换一个Transformer层与一个构象层导致进一步增益的词汇度量。实验结果表明,在语音度量,1.5%的词汇度量,和2.3%的句法度量相比,实现了一个相对的改进的基准方法的CPC-小,这是训练LibriSpeech 460小时的数据。我们在这个挑战中取得了最好的结果与语法度量。
英文摘要:
We present a system for the Zero Resource Speech Challenge 2021, which combines a Contrastive Predictive Coding (CPC) with deep cluster. In deep cluster, we first prepare pseudo-labels obtained by clustering the outputs of a CPC network with k-means. Then, we train an additional autoregressive model to classify the previously obtained pseudo-labels in a supervised manner. Phoneme discriminative representation is achieved by executing the second-round clustering with the outputs of the final layer of the autoregressive model. We show that replacing a Transformer layer with a Conformer layer leads to a further gain in a lexical metric. Experimental results show that a relative improvement of 35% in a phonetic metric, 1.5% in the lexical metric, and 2.3% in a syntactic metric are achieved compared to a baseline method of CPC-small which is trained on LibriSpeech 460h data. We achieve top results in this challenge with the syntactic metric.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。