使用采样频率无关卷积层的可调节令牌时间分辨率的神经音频编解码器
Neural Audio Codec with Adjustable Token Temporal Resolution Using Sampling-Frequency-Independent Convolutional Layers
AI总结：
提出一种机制，通过采样频率无关卷积层使单个神经音频编解码器支持多种令牌时间分辨率，在环境声音重建中优于切换专用层的基线模型。
摘要：
从神经音频编解码器（NAC）获得的离散令牌已被用作音频生成和理解模型中的紧凑表示。在这种基于令牌的系统中，令牌时间分辨率（TTR）定义为相邻令牌帧之间的时间间隔，它很重要，因为它控制着表示快速声学事件与减少令牌序列长度之间的权衡。然而，大多数NAC在单个TTR下训练，并且需要为每个TTR单独训练。本文提出一种机制，通过使用采样频率无关的卷积层，使单个NAC能够在多个TTR下运行。该机制将TTR视为令牌序列的采样周期，并从共享参数集生成依赖于TTR的卷积核，同时针对每个TTR调整核大小和步长。我们将该机制集成到Descript Audio Codec中，保持量化器不变。在环境声音重建上的实验表明，所提出的模型优于为每个TTR切换专用层的单模型基线。
英文摘要：
Discrete tokens obtained from neural audio codecs (NACs) have been used as compact representations in audio generation and understanding models. In such token-based systems, token temporal resolution (TTR), defined as the time interval between adjacent token frames, is important because it controls the trade-off between representing rapid acoustic events and reducing token-sequence length. However, most NACs are trained at a single TTR and require separate training for each TTR. This paper proposes a mechanism that enables a single NAC to operate at multiple TTRs using sampling-frequency-independent convolutional layers. The mechanism regards TTR as the sampling period of the token sequence and generates TTR-dependent convolutional kernels from a shared parameter set, while adjusting the kernel size and stride for each TTR. We incorporate the mechanism into Descript Audio Codec, leaving the quantizer unchanged. Experiments on environmental sound reconstruction show that the proposed model outperforms a single-model baseline that switches TTR-specific layers for each TTR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
