UniCodec：具有单一域自适应码本的统一音频编解码器
UniCodec: Unified Audio Codec with Single Domain-Adaptive Codebook
AI总结：
UniCodec提出分区域自适应码本与域专家混合策略，以单一码本统一处理语音、音乐和声音，通过自监督掩码预测增强语义，重建性能超越现有统一及特定域编解码器。
摘要：
音频语言模型的出现得益于神经音频编解码器，这些编解码器在连续波形与离散令牌之间建立了关键映射，使其与语言模型范式兼容。从多层残差向量量化器到单层量化器的演进趋势有利于语言自回归解码。然而，通过单一码本处理多域音频信号的能力仍受限于域间分布差异。在本工作中，我们引入了UniCodec，一种具有单一码本以支持多域音频数据（包括语音、音乐和声音）的统一音频编解码器。为实现这一目标，我们提出了一种分区域自适应码本方法和域专家混合策略，以捕捉每个音频域的独特特征。此外，为在不使用辅助模块的情况下丰富编解码器的语义密度，我们提出了一种自监督掩码预测建模方法。全面的主观和客观评估表明，UniCodec在三个音频域中实现了出色的音频重建性能，优于现有的具有单一码本的统一神经编解码器，并且在声学和语义表示能力上甚至超越了最先进的特定域编解码器。
英文摘要：
The emergence of audio language models is empowered by neural audio codecs, which establish critical mappings between continuous waveforms and discrete tokens compatible with language model paradigms. The evolutionary trends from multi-layer residual vector quantizer to single-layer quantizer are beneficial for language-autoregressive decoding. However, the capability to handle multi-domain audio signals through a single codebook remains constrained by inter-domain distribution discrepancies. In this work, we introduce UniCodec, a unified audio codec with a single codebook to support multi-domain audio data, including speech, music, and sound. To achieve this, we propose a partitioned domain-adaptive codebook method and domain Mixture-of-Experts strategy to capture the distinct characteristics of each audio domain. Furthermore, to enrich the semantic density of the codec without auxiliary modules, we propose a self-supervised mask prediction modeling approach. Comprehensive objective and subjective evaluations demonstrate that UniCodec achieves excellent audio reconstruction performance across the three audio domains, outperforming existing unified neural codecs with a single codebook, and even surpasses state-of-the-art domain-specific codecs on both acoustic and semantic representation capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
