arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2008.01393cs.SDcs.LGeess.AS
收藏

神经颗粒声音合成

Neural Granular Sound Synthesis

IRCAM(法国声学音乐协调及研究中心);CNRS(法国国家科学研究中心);The University of Tokyo(东京大学);RIKEN(日本理化学研究所)

Adrien Bitton, Philippe Esling, Tatsuya Harada

摘要:

颗粒声音合成是一种基于重排小波形窗口序列的流行音频生成技术。为了控制合成,在一个给定的语料库中的所有颗粒进行分析,通过一组声学描述符。这提供了反映跨颗粒的某种形式的局部相似性的表示。然而,该颗粒空间的质量受到描述符的质量的约束。它的遍历不是连续可逆的信号,并不呈现任何结构化的时间性。我们证明了生成神经网络可以实现粒度合成,同时减轻其大部分缺点。我们有效地取代其音频描述符的基础上学习的概率潜在空间与变分自动编码器。在这种设置中,学习的颗粒空间是可逆的,这意味着我们可以在遍历其维度时连续合成声音。这也意味着原始的谷物不是为了合成而储存的。我们的方法的另一个主要优点是通过在排列的颗粒序列上训练更高级别的时间嵌入来学习这个潜在空间内的结构化路径。该模型可以应用于许多类型的图书馆,包括音高音符或非音高鼓和环境噪声。我们报告实验上的共同颗粒合成过程以及新的,如条件采样和变形。

英文摘要:

Granular sound synthesis is a popular audio generation technique based on rearranging sequences of small waveform windows. In order to control the synthesis, all grains in a given corpus are analyzed through a set of acoustic descriptors. This provides a representation reflecting some form of local similarities across the grains. However, the quality of this grain space is bound by that of the descriptors. Its traversal is not continuously invertible to signal and does not render any structured temporality. We demonstrate that generative neural networks can implement granular synthesis while alleviating most of its shortcomings. We efficiently replace its audio descriptor basis by a probabilistic latent space learned with a Variational Auto-Encoder. In this setting the learned grain space is invertible, meaning that we can continuously synthesize sound when traversing its dimensions. It also implies that original grains are not stored for synthesis. Another major advantage of our approach is to learn structured paths inside this latent space by training a higher-level temporal embedding over arranged grain sequences. The model can be applied to many types of libraries, including pitched notes or unpitched drums and environmental noises. We report experiments on the common granular synthesis processes as well as novel ones such as conditional sampling and morphing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑