低帧率语音编解码器：一种为快速高质量语音LLM训练与推理设计的编解码器
Low Frame-rate Speech Codec: a Codec Designed for Fast High-quality Speech LLM Training and Inference
AI总结：
针对现有音频编解码器高帧率导致语音LLM训练推理慢的问题，提出结合有限标量量化与对抗训练的低帧率语音编解码器LFSC，在低比特率下实现高质量压缩，可使文本转语音推理提速约三倍且质量相当。
摘要：
大语言模型（LLM）通过可将音频转换为离散token的音频编解码器，极大推动了音频处理领域的发展，使得语言建模技术能够应用于音频数据。然而，音频编解码器通常以高帧率运行，导致训练与推理速度缓慢，对于自回归模型而言尤为明显。为解决这一难题，我们提出低帧率语音编解码器（LFSC）：这是一种神经音频编解码器，结合有限标量量化与面向大语音语言模型的对抗训练，在1.89 kbps比特率、21.5帧每秒的配置下实现高质量音频压缩。实验表明，这种新型编解码器可使基于LLM的文本转语音模型推理速度提升约三倍，同时提升可懂度，生成质量与过往模型相当。
英文摘要：
Large language models (LLMs) have significantly advanced audio processing through audio codecs that convert audio into discrete tokens, enabling the application of language modeling techniques to audio data. However, audio codecs often operate at high frame rates, resulting in slow training and inference, especially for autoregressive models. To address this challenge, we present the Low Frame-rate Speech Codec (LFSC): a neural audio codec that leverages finite scalar quantization and adversarial training with large speech language models to achieve high-quality audio compression with a 1.89 kbps bitrate and 21.5 frames per second. We demonstrate that our novel codec can make the inference of LLM-based text-to-speech models around three times faster while improving intelligibility and producing quality comparable to previous models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
