arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.02776cs.SDcs.AIcs.MM
收藏

UniSRCodec:统一且低比特率单码本编解码器,支持子带重建

UniSRCodec: Unified and Low-Bitrate Single Codebook Codec with Sub-Band Reconstruction

Shenzhen International Graduate School, Tsinghua University, Shenzhen, China(清华大学深圳国际研究生院);ModelBest Inc., Beijing, China(ModelBest公司)

Zhisheng Zhang, Xiang Li, Yixuan Zhou, Jing Peng, Shengbo Cai, Guoyang Zeng, Zhiyong Wu

AI总结:

UniSRCodec是一种低比特率单码本编解码器,通过子带重建技术实现高保真度压缩,优于现有单码本方法。

摘要:

神经音频编解码器(NACs)通过进行紧凑压缩和重建来减少传输开销,旨在弥合连续信号和离散信号之间的差距。现有的NACs可分为两类:多码本编解码器和单码本编解码器。多码本编解码器面临结构复杂性和适应下游任务的困难,而单码本编解码器虽然结构更简单,但存在低保真度、对统一音频建模效果不佳以及无法支持高频音频建模的问题。我们提出了UniSRCodec,一种能够支持高采样率、低带宽、高保真度和统一的单码本编解码器。我们分析了基于波形的压缩效率低下,并引入利用梅尔频谱图的时间和频率压缩方法,并与Vocoder配合以恢复原始音频的相位信息。此外,我们提出了一种子带重建技术,以在低频和高频带实现高质量的压缩。主观和客观实验结果表明,UniSRCodec在跨领域单码本编解码器中仅以40个token的速率实现了最先进的性能,其重建质量可与某些多码本方法相媲美。我们的演示页面可在https://wxzyd123.github.io/unisrcodec上找到。

英文摘要:

Neural Audio Codecs (NACs) can reduce transmission overhead by performing compact compression and reconstruction, which also aim to bridge the gap between continuous and discrete signals. Existing NACs can be divided into two categories: multi-codebook and single-codebook codecs. Multi-codebook codecs face challenges such as structural complexity and difficulty in adapting to downstream tasks, while single-codebook codecs, though structurally simpler, suffer from low-fidelity, ineffective modeling of unified audio, and an inability to support modeling of high-frequency audio. We propose the UniSRCodec, a single-codebook codec capable of supporting high sampling rate, low-bandwidth, high fidelity, and unified. We analyze the inefficiency of waveform-based compression and introduce the time and frequency compression method using the Mel-spectrogram, and cooperate with a Vocoder to recover the phase information of the original audio. Moreover, we propose a sub-band reconstruction technique to achieve high-quality compression across both low and high frequency bands. Subjective and objective experimental results demonstrate that UniSRCodec achieves state-of-the-art (SOTA) performance among cross-domain single-codebook codecs with only a token rate of 40, and its reconstruction quality is comparable to that of certain multi-codebook methods. Our demo page is available at https://wxzyd123.github.io/unisrcodec.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑