使用可微分DSP的快速、高质量且参数高效的发音合成
Fast, High-Quality and Parameter-Efficient Articulatory Synthesis using Differentiable DSP
AI总结：
本文提出一种结合EMA、F0和响度的参数高效DDSP发音声码器，通过谐波/噪声平衡技术和多分辨率对抗损失，以0.4M参数实现比SOTA更快、更高质量的语音合成。
摘要：
诸如电磁发音造影（EMA）等发音轨迹提供了声道滤波器的低维表示，并已被用作语音合成的自然、有依据的特征。可微分数字信号处理（DDSP）是一种参数高效的音频合成框架。因此，将低维EMA特征与DDSP相结合可以显著提高语音合成的计算效率。在本文中，我们提出了一种快速、高质量且参数高效的DDSP发音声码器，它可以从EMA、F0和响度合成语音。我们采用了多种技术来解决谐波/噪声不平衡问题，并添加了多分辨率对抗损失以获得更好的合成质量。我们的模型实现了6.67%的转录词错误率（WER）和3.74的平均意见得分（MOS），与最先进（SOTA）基线相比分别改善了1.63%和0.16。我们的DDSP声码器在CPU上推理时比基线快4.9倍，并且仅用0.4M参数即可生成质量相当的语音，而SOTA需要9M参数。
英文摘要：
Articulatory trajectories like electromagnetic articulography (EMA) provide a low-dimensional representation of the vocal tract filter and have been used as natural, grounded features for speech synthesis. Differentiable digital signal processing (DDSP) is a parameter-efficient framework for audio synthesis. Therefore, integrating low-dimensional EMA features with DDSP can significantly enhance the computational efficiency of speech synthesis. In this paper, we propose a fast, high-quality, and parameter-efficient DDSP articulatory vocoder that can synthesize speech from EMA, F0, and loudness. We incorporate several techniques to solve the harmonics / noise imbalance problem, and add a multi-resolution adversarial loss for better synthesis quality. Our model achieves a transcription word error rate (WER) of 6.67% and a mean opinion score (MOS) of 3.74, with an improvement of 1.63% and 0.16 compared to the state-of-the-art (SOTA) baseline. Our DDSP vocoder is 4.9x faster than the baseline on CPU during inference, and can generate speech of comparable quality with only 0.4M parameters, in contrast to the 9M parameters required by the SOTA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
