Audio codecs are typically transform-domain based and efficiently code stationary audio signals, but they struggle with speech and signals containing dense transient events such as applause. Specifically, with these two classes of signals as examples, we demonstrate a technique for restoring audio from coding noise based on generative adversarial networks (GAN). A primary advantage of the proposed GAN-based coded audio enhancer is that the method operates end-to-end directly on decoded audio samples, eliminating the need to design any manually-crafted frontend. Furthermore, the enhancement approach described in this paper can improve the sound quality of low-bit rate coded audio without any modifications to the existent standard-compliant encoders. Subjective tests illustrate that the proposed enhancer improves the quality of speech and difficult to code applause excerpts significantly.
Codec-SUPERB @ SLT 2024: A lightweight benchmark for neural audio codec models
National Taiwan University（台湾大学）；Massachusetts Institute of Technology（麻省理工学院）；Microsoft Corporation（微软公司）；Meta；The Chinese University of Hong Kong（香港中文大学）；Carnegie Mellon University（卡内基梅隆大学）
ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and Speech
Carnegie Mellon University（卡内基梅隆大学）；Renmin University of China（中国人民大学）；Nanyang Technological University（南洋理工大学）；Tokyo Metropolitan University（东京都立大学）；Fudan University（复旦大学）；University of Chicago（芝加哥大学）；National Taiwan University（台湾大学）；Massachusetts Institute of Technology（麻省理工学院）
UniCodec: Unified Audio Codec with Single Domain-Adaptive Codebook
National University of Singapore（新加坡国立大学）；Tongyi Speech Lab（通义语音实验室）；Shanghai Jiao Tong University（上海交通大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）
Collaboration AI, Cisco Systems, Inc.（Collaboration AI，Cisco系统公司）；Hebrew University of Jerusalem, Israel（耶路撒冷希伯来大学）；University of Illinois Urbana-Champaign, United States（伊利诺伊大学厄巴纳-香槟分校）；Carnegie Mellon University, United States（卡内基梅隆大学）
APCodec: A Neural Audio Codec with Parallel Amplitude and Phase Spectrum Encoding and Decoding
University of Science and Technology of China（中国科学技术大学）；National Engineering Research Center of Speech and Language Information Processing（国家语言语音信息处理工程研究中心）
Large-scale mobile communication systems tend to contain legacy transmission channels with narrowband bottlenecks, resulting in characteristic "telephone-quality" audio. While higher quality codecs exist, due to the scale and heterogeneity of the networks, transmitting higher sample rate audio with modern high-quality audio codecs can be difficult in practice. This paper proposes an approach where a communication node can instead extend the bandwidth of a band-limited incoming speech signal that may have been passed through a low-rate codec. To this end, we propose a WaveNet-based model condit…
Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model
Hong Kong University of Science and Technology（香港科技大学）；University of Science and Technology Beijing（北京科技大学）；Hong Kong Baptist University（香港浸会大学）；Microsoft（微软公司）；Chinese University of Hong Kong（香港中文大学）
针对现有音频 LLM 编解码器因压缩设计导致的语义缺陷问题，提出 X-Codec 方法，通过在 RVQ 前后融入语义特征与损失，显著降低语音合成 WER 并扩展至非语音任务，提升音频生成性能。
Codecfake: An Initial Dataset for Detecting LLM-based Deepfake Audio
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Information and Communication Engineering, Communication University of China（中国传媒大学信息与通信工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）；Beijing National Research Center for Information Science and Technology, Tsinghua University（清华大学北京信息科学与技术国家研究中心）；Shanghai Polytechnic University（上海第二工业大学）
