arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1904.09407cs.CL
收藏

基于GAN的自模仿语音反馈生成及其在语音训练中的应用

Self-imitating Feedback Generation Using GAN for Computer-Assisted Pronunciation Training

Seoul National University(首尔大学)

Seung Hee Yang, Minhwa Chung

摘要:

在计算机辅助语音训练中,自我模仿反馈是一种有效的、对学习者友好的方法。本族语中的声学特征被提取并移植到学习者自己的语音输入中,并作为纠正反馈反馈给学习者。以往的研究主要集中在基于PSOLA算法的韵律移植语音转换技术上。基于母语和非母语语音谱图中的视觉差异,我们研究了应用GAN通过对抗训练利用生成器的能力来生成自我模仿反馈。由于这种映射是高度欠约束的,我们还采用循环一致性损失来鼓励输出保持本地和非本地话语共享的全局结构。在对97,200幅由母语和非母语韩国人产生的短话语的频谱图图像进行训练后,生成器能够成功地将非母语频谱图输入转换为具有自我模仿反馈特性的频谱图。此外,转换后的声谱图显示出无法通过韵律移植获得的分段校正。感知测试比较我们的方法与基线PSOLA方法的自我模仿和纠正能力表明,循环一致性损失的生成方法是有前途的。

英文摘要:

Self-imitating feedback is an effective and learner-friendly method for non-native learners in Computer-Assisted Pronunciation Training. Acoustic characteristics in native utterances are extracted and transplanted onto learner's own speech input, and given back to the learner as a corrective feedback. Previous works focused on speech conversion using prosodic transplantation techniques based on PSOLA algorithm. Motivated by the visual differences found in spectrograms of native and non-native speeches, we investigated applying GAN to generate self-imitating feedback by utilizing generator's ability through adversarial training. Because this mapping is highly under-constrained, we also adopt cycle consistency loss to encourage the output to preserve the global structure, which is shared by native and non-native utterances. Trained on 97,200 spectrogram images of short utterances produced by native and non-native speakers of Korean, the generator is able to successfully transform the non-native spectrogram input to a spectrogram with properties of self-imitating feedback. Furthermore, the transformed spectrogram shows segmental corrections that cannot be obtained by prosodic transplantation. Perceptual test comparing the self-imitating and correcting abilities of our method with the baseline PSOLA method shows that the generative approach with cycle consistency loss is promising.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑