arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.13438eess.AScs.AIcs.SDstat.ML
收藏

高保真神经音频压缩

High Fidelity Neural Audio Compression

Meta AI

Alexandre Défossez, Jade Copet, Gabriel Synnaeve, Yossi Adi

摘要:

我们介绍了一个国家的最先进的实时,高保真,音频编解码器利用神经网络。它包括一个流编码器-解码器架构,具有以端到端方式训练的量化潜在空间。我们通过使用单个多尺度谱图对手来简化和加速训练,该对手可以有效地减少伪影并生成高质量的样本。我们引入了一种新的损失平衡器机制来稳定训练:损失的权重现在定义了它应该代表的整体梯度的分数,从而将这个超参数的选择与损失的典型规模解耦。最后,我们研究了如何使用轻量级的Transformer模型来进一步压缩所获得的表示高达40%,同时保持比实时更快。我们提供了一个详细的描述所提出的模型的关键设计选择,包括:训练目标,架构的变化和各种感知损失函数的研究。我们提出了一个广泛的主观评价(MUSHRA测试)连同消融研究的带宽和音频域的范围内,包括语音,嘈杂的混响语音,和音乐。考虑到24 kHz单声道和48 kHz立体声音频,我们的方法优于所有评估设置的基线方法。代码和模型可在github.com/facebookresearch/encodec上获得。

英文摘要:

We introduce a state-of-the-art real-time, high-fidelity, audio codec leveraging neural networks. It consists in a streaming encoder-decoder architecture with quantized latent space trained in an end-to-end fashion. We simplify and speed-up the training by using a single multiscale spectrogram adversary that efficiently reduces artifacts and produce high-quality samples. We introduce a novel loss balancer mechanism to stabilize training: the weight of a loss now defines the fraction of the overall gradient it should represent, thus decoupling the choice of this hyper-parameter from the typical scale of the loss. Finally, we study how lightweight Transformer models can be used to further compress the obtained representation by up to 40%, while staying faster than real time. We provide a detailed description of the key design choices of the proposed model including: training objective, architectural changes and a study of various perceptual loss functions. We present an extensive subjective evaluation (MUSHRA tests) together with an ablation study for a range of bandwidths and audio domains, including speech, noisy-reverberant speech, and music. Our approach is superior to the baselines methods across all evaluated settings, considering both 24 kHz monophonic and 48 kHz stereophonic audio. Code and models are available at github.com/facebookresearch/encodec.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑