arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.08514cs.SDcs.AIeess.AS
收藏

Apollo:面向高质量音频修复的频带序列建模

Apollo: Band-sequence Modeling for High-Quality Audio Restoration

Tsinghua University(清华大学);Tencent AI Lab(腾讯AI实验室)

Kai Li, Yi Luo

AI总结:

本文提出高采样率音频修复生成模型 Apollo,通过显式频带分割与频带关系建模,在多比特率和多流派音乐修复中优于 SR-GAN。

摘要:

音频修复在现代社会中变得日益重要,这不仅是因为先进播放设备带来了对高质量听觉体验的需求,也因为生成式音频模型不断增强的能力需要高保真音频。通常,音频修复被定义为从受损输入预测无失真音频的任务,并常使用 GAN 框架进行训练,以平衡感知质量与失真。由于音频退化主要集中在中高频范围,尤其是由编解码器造成的退化,一项关键挑战在于设计一种既能保留低频信息、又能准确重建高质量中高频内容的生成器。受高采样率音乐分离、语音增强和音频编解码器模型最新进展的启发,我们提出 Apollo,这是一种专为高采样率音频修复设计的生成模型。Apollo 采用显式频带分割模块来建模不同频带之间的关系,从而实现更连贯、更高质量的修复音频。在 MUSDB18-HQ 和 MoisesDB 数据集上的评估表明,Apollo 在不同比特率和音乐流派下始终优于现有 SR-GAN 模型,尤其擅长处理包含多种乐器与人声混合的复杂场景。Apollo 在保持计算效率的同时显著提升了音乐修复质量。Apollo 的源代码已公开发布于 https://github.com/JusperLee/Apollo。

英文摘要:

Audio restoration has become increasingly significant in modern society, not only due to the demand for high-quality auditory experiences enabled by advanced playback devices, but also because the growing capabilities of generative audio models necessitate high-fidelity audio. Typically, audio restoration is defined as a task of predicting undistorted audio from damaged input, often trained using a GAN framework to balance perception and distortion. Since audio degradation is primarily concentrated in mid- and high-frequency ranges, especially due to codecs, a key challenge lies in designing a generator capable of preserving low-frequency information while accurately reconstructing high-quality mid- and high-frequency content. Inspired by recent advancements in high-sample-rate music separation, speech enhancement, and audio codec models, we propose Apollo, a generative model designed for high-sample-rate audio restoration. Apollo employs an explicit frequency band split module to model the relationships between different frequency bands, allowing for more coherent and higher-quality restored audio. Evaluated on the MUSDB18-HQ and MoisesDB datasets, Apollo consistently outperforms existing SR-GAN models across various bit rates and music genres, particularly excelling in complex scenarios involving mixtures of multiple instruments and vocals. Apollo significantly improves music restoration quality while maintaining computational efficiency. The source code for Apollo is publicly available at https://github.com/JusperLee/Apollo.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑