神经音频编解码器中的源解耦学习
Learning Source Disentanglement in Neural Audio Codec
AI总结：
针对现有神经音频编解码器忽略不同声音域差异导致建模复杂和生成可控性差的问题，提出结合编码与分离的SD-Codec，实现不同源的码本解耦，在保持重合成质量的同时增强可解释性与生成可控性。
摘要：
神经音频编解码器通过将连续音频信号高效转换为离散令牌，显著推动了音频压缩技术的发展。这些编解码器能保留高质量声音，并通过在令牌上训练的生成模型实现复杂的声音生成。然而，现有神经编解码器模型通常在大型、未区分的音频数据集上训练，忽略了语音、音乐和环境音效等声音域之间的关键差异。这一疏忽使数据建模复杂化，并为声音生成的可控性带来额外挑战。为解决这些问题，我们提出源解耦神经音频编解码器（SD-Codec），这是一种结合音频编码和源分离的新方法。通过联合学习音频重合成和分离，SD-Codec将不同域的音频信号明确分配到不同的码本（离散表示集）中。实验结果表明，SD-Codec不仅保持了具有竞争力的重合成质量，而且在分离结果的支持下，实现了潜在空间中不同源的成功解耦，从而增强了音频编解码器的可解释性，并为音频生成过程提供了更精细控制的可能。
英文摘要：
Neural audio codecs have significantly advanced audio compression by efficiently converting continuous audio signals into discrete tokens. These codecs preserve high-quality sound and enable sophisticated sound generation through generative models trained on these tokens. However, existing neural codec models are typically trained on large, undifferentiated audio datasets, neglecting the essential discrepancies between sound domains like speech, music, and environmental sound effects. This oversight complicates data modeling and poses additional challenges to the controllability of sound generation. To tackle these issues, we introduce the Source-Disentangled Neural Audio Codec (SD-Codec), a novel approach that combines audio coding and source separation. By jointly learning audio resynthesis and separation, SD-Codec explicitly assigns audio signals from different domains to distinct codebooks, sets of discrete representations. Experimental results indicate that SD-Codec not only maintains competitive resynthesis quality but also, supported by the separation results, demonstrates successful disentanglement of different sources in the latent space, thereby enhancing interpretability in audio codec and providing potential finer control over the audio generation process.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
