High-Quality, Low-Delay Music Coding in the Opus Codec

A Full-Bandwidth Audio Codec With Low Complexity And Very Low Delay

We propose an audio codec that addresses the low-delay requirements of some applications such as network music performance. The codec is based on the modified discrete cosine transform (MDCT) with very short frames and uses gain-shape quantization to preserve the spectral envelope. The short frame sizes required for low delay typically hinder the performance of transform codecs. However, at 96 kbit/s and with only 4 ms algorithmic delay, the proposed codec out-performs the ULD codec operating at the same rate. The total complexity of the codec is small, at only 17 WMOPS for real-time operation…

Efficient And Scalable Neural Residual Waveform Coding With Collaborative Quantization

Scalability and efficiency are desired in neural speech codecs, which supports a wide range of bitrates for applications on various devices. We propose a collaborative quantization (CQ) scheme to jointly learn the codebook of LPC coefficients and the corresponding residuals. CQ does not simply shoehorn LPC to a neural network, but bridges the computational capacity of advanced neural network models and traditional, yet efficient and domain-specific digital signal processing methods in an integrated manner. We demonstrate that CQ achieves much higher quality than its predecessor at 9 kbps with …

V2Meow：通过视频到音乐生成随视觉节拍喵叫

V2Meow: Meowing to the Visual Beat via Video-to-Music Generation

本文提出V2Meow，一个基于多阶段自回归模型的视频到音乐生成系统，通过直接学习视频与音乐的全局对齐特征，在零样本条件下实现高质量音乐生成，并在视觉-音频对应性和音频质量上优于现有系统。

SemantiCodec：面向通用声音的超低比特率语义音频编解码器

SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound

针对传统音频编解码器高比特率、窄域及缺乏语义线索的问题，提出具有双编码器架构的SemantiCodec，可在超低比特率下压缩多类型音频且语义信息更丰富，重建质量优于现有技术。

Perceptual Vector Quantization For Video Coding

This paper applies energy conservation principles to the Daala video codec using gain-shape vector quantization to encode a vector of AC coefficients as a length (gain) and direction (shape). The technique originates from the CELT mode of the Opus audio codec, where it is used to conserve the spectral envelope of an audio signal. Conserving energy in video has the potential to preserve textures rather than low-passing them. Explicitly quantizing a gain allows a simple contrast masking model with no signaling cost. Vector quantizing the shape keeps the number of degrees of freedom the same as s…

STCTS: 通过显式文本-语调-音色分解实现超低比特率语音的生成语义压缩

STCTS: Generative Semantic Compression for Ultra-Low Bitrate Speech via Explicit Text-Prosody-Timbre Decomposition

STCTS通过显式分解语音为语言内容、语调和音色，实现80bps下的高质量超低比特率语音压缩，兼顾效率与质量。