利用生成对抗网络增强含间隙语音音频信号
Enhancing Gappy Speech Audio Signals with Generative Adversarial Networks
AI总结：
本文利用生成对抗网络，通过将音频转化为梅尔频谱图进行图像修复，再经Parallel-WaveGAN声码器还原，实现长达320毫秒语音间隙的近乎实时高质量再生，在240毫秒间隙上达到3.737的MOS评分。
摘要：
音频中的间隙、丢失和损坏的短片段是一个常见问题，尤其是在语音中出现时尤为恼人。本文利用机器学习来再生语音音频信号中长达320毫秒的间隙。通过将音频转换为梅尔频谱图，并利用图像修复技术再生间隙，从而将音频再生转化为图像再生。随后，使用Parallel-WaveGAN声码器将完整的梅尔频谱图转换回音频，并集成到音频流中。使用取自公开可用的LJSpeech数据集的1300个时长在1至10秒之间的口语音频片段样本，我们的结果表明，在使用配备GPU的系统上，通过生成对抗网络（GANs）几乎可以实时地再生音频间隙。正如预期的那样，音频中的间隙越小，填充间隙的质量越好。在240毫秒的间隙上，表现最佳的模型的平均意见得分（MOS）为3.737（评分范围从1（最差）到5（最好）），这足以让人类感知为接近不间断的人类语音。
英文摘要：
Gaps, dropouts and short clips of corrupted audio are a common problem and particularly annoying when they occur in speech. This paper uses machine learning to regenerate gaps of up to 320ms in an audio speech signal. Audio regeneration is translated into image regeneration by transforming audio into a Mel-spectrogram and using image in-painting to regenerate the gaps. The full Mel-spectrogram is then transferred back to audio using the Parallel-WaveGAN vocoder and integrated into the audio stream. Using a sample of 1300 spoken audio clips of between 1 and 10 seconds taken from the publicly-available LJSpeech dataset our results show regeneration of audio gaps in close to real time using GANs with a GPU equipped system. As expected, the smaller the gap in the audio, the better the quality of the filled gaps. On a gap of 240ms the average mean opinion score (MOS) for the best performing models was 3.737, on a scale of 1 (worst) to 5 (best) which is sufficient for a human to perceive as close to uninterrupted human speech.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
