Puffin:面向普通设备的全带语音基音同步神经波形生成
Puffin: pitch-synchronous neural waveform generation for fullband speech on modest devices
AI总结:
针对低功耗设备神经声码器复杂度过高的问题,研究提出Puffin神经声码器,结合现代与老一代声码器技术,采用基音同步ISTFT和对抗训练,以极低计算量实现了媲美HiFi-GAN的48kHz语音合成质量。
摘要:
我们提出了一款专为低功耗替代与增强沟通设备设计的神经声码器。通过将现代优秀声码器的设计元素与老一代技术的成熟思路相结合,我们的系统能够在神经声码器复杂度原本高到无法运行的设备上,生成48kHz的高质量合成语音。该系统采用可微基音同步叠加相加算法进行对抗训练,并依托基音同步逆短时傅里叶变换(ISTFT)生成语音样本以降低复杂度。我们的系统仅用极少的计算量,就达到了与强基线(HiFi-GAN)相当的质量。文中还呈现了感知评估结果以及系统复杂度分析。
英文摘要:
We present a neural vocoder designed with low-powered Alternative and Augmentative Communication devices in mind. By combining elements of successful modern vocoders with established ideas from an older generation of technology, our system is able to produce high quality synthetic speech at 48kHz on devices where neural vocoders are otherwise prohibitively complex. The system is trained adversarially using differentiable pitch synchronous overlap add, and reduces complexity by relying on pitch synchronous Inverse Short-Time Fourier Transform (ISTFT) to generate speech samples. Our system achieves comparable quality with a strong (HiFi-GAN) baseline while using only a fraction of the compute. We present results of a perceptual evaluation as well as an analysis of system complexity.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。