DualCodec：一种用于语音生成的低帧率、语义增强型神经音频编解码器
DualCodec: A Low-Frame-Rate, Semantically-Enhanced Neural Audio Codec for Speech Generation
AI总结：
针对神经音频编解码器帧率与音质的权衡问题，本研究提出双流编码的DualCodec，整合自监督与波形表示增强语义，实现低帧率下的高音质，提升语音生成效率，性能优于现有主流编解码器。
摘要：
神经音频编解码器是基于语言模型（LM）的语音生成的核心基础模块。通常而言，帧率与音频质量之间存在权衡关系。本研究提出了一种低帧率、语义增强的编解码器模型。现有方法将语义丰富的自监督（SSL）表示蒸馏到第一层编解码器令牌中。本研究提出DualCodec，这是一种双流编码方法，在端到端编解码器框架内整合了SSL表示与波形表示。在该架构下，DualCodec增强了第一层编解码器中的语义信息，使编解码器系统能够在低帧率运行的同时保持高音频质量。需要注意的是，低帧率编解码器可提升语音生成的效率。在音频编解码和语音生成任务上的实验结果证实，与Mimi Codec、SpeechTokenizer、DAC、Encodec等最先进的编解码器系统相比，所提出的DualCodec具有更优的性能。演示页面可访问：https://dualcodec.github.io，代码可访问：https://github.com/jiaqili3/DualCodec
英文摘要：
Neural audio codecs form the foundational building blocks for language model (LM)-based speech generation. Typically, there is a trade-off between frame rate and audio quality. This study introduces a low-frame-rate, semantically enhanced codec model. Existing approaches distill semantically rich self-supervised (SSL) representations into the first-layer codec tokens. This work proposes DualCodec, a dual-stream encoding approach that integrates SSL and waveform representations within an end-to-end codec framework. In this setting, DualCodec enhances the semantic information in the first-layer codec and enables the codec system to maintain high audio quality while operating at a low frame rate. Note that a low-frame-rate codec improves the efficiency of speech generation. Experimental results on audio codec and speech generation tasks confirm the effectiveness of the proposed DualCodec compared to state-of-the-art codec systems, such as Mimi Codec, SpeechTokenizer, DAC, and Encodec. Demos are available at: https://dualcodec.github.io, code is available at: https://github.com/jiaqili3/DualCodec
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
