arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.05050cs.SDcs.LGeess.ASeess.SP
收藏

神经波形合成

Neural Waveshaping Synthesis

Queen Mary University of London(伦敦玛丽女王大学)

Ben Hayes, Charalampos Saitis, György Fazekas

摘要:

我们提出了神经波整形单元(NEWT):一种新颖的,轻量级的,完全因果关系的神经音频合成方法,直接在波形域中操作,伴随着高效的CPU推理优化(FastNEWT)。NEWT使用具有周期性激活的时间分布多层感知器来隐式地学习对目标音色的特征进行编码的非线性传递函数。一旦训练好,NEWT就可以通过对其输入和输出信号进行简单的仿射变换来产生复杂的音色演变。我们将NEWT与可区分的噪声合成器和混响配对,发现它能够在F0和响度特征的条件下仅用260k总模型参数生成逼真的乐器表演。我们将我们的方法与多刺激听力测试和Fréchet音频距离的最先进基准进行了比较,发现它在测试的音色域中具有竞争力。我们的方法在生成速度方面显著优于基准测试,并在有和没有FastNEWT的消费CPU上实现了实时性能,这表明它是未来创意声音设计工具的可行基础。

英文摘要:

We present the Neural Waveshaping Unit (NEWT): a novel, lightweight, fully causal approach to neural audio synthesis which operates directly in the waveform domain, with an accompanying optimisation (FastNEWT) for efficient CPU inference. The NEWT uses time-distributed multilayer perceptrons with periodic activations to implicitly learn nonlinear transfer functions that encode the characteristics of a target timbre. Once trained, a NEWT can produce complex timbral evolutions by simple affine transformations of its input and output signals. We paired the NEWT with a differentiable noise synthesiser and reverb and found it capable of generating realistic musical instrument performances with only 260k total model parameters, conditioned on F0 and loudness features. We compared our method to state-of-the-art benchmarks with a multi-stimulus listening test and the Fréchet Audio Distance and found it performed competitively across the tested timbral domains. Our method significantly outperformed the benchmarks in terms of generation speed, and achieved real-time performance on a consumer CPU, both with and without FastNEWT, suggesting it is a viable basis for future creative sound design tools.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑