VampNet：基于掩码声学 token 建模的音乐生成
VampNet: Music Generation via Masked Acoustic Token Modeling
AI总结：
本文提出非自回归双向 transformer 模型 VampNet，通过可变掩码声学 token 建模与灵活提示，在36次采样内生成高保真音乐，并支持压缩、修复、续写和变奏循环等创作任务。
摘要：
我们提出了 VampNet，这是一种用于音乐合成、压缩、音频修复和变奏的掩码声学 token 建模方法。我们在训练过程中采用可变掩码调度，使模型能够在推理时通过应用多种掩码方式（称为提示）采样出连贯的音乐。VampNet 是非自回归的，利用双向 transformer 架构，在前向传播中关注所有 token。仅需 36 次采样迭代，VampNet 即可生成连贯且高保真的音乐波形。我们展示了通过以多种方式提示 VampNet，可将其应用于音乐压缩、修复、外延补全、续写以及带变奏的循环（即 vamping）等任务。在适当提示下，VampNet 能够保持音乐的风格、流派、配器及其他高层特征。这种灵活的提示能力使 VampNet 成为强大的音乐协同创作工具。代码和音频样本已在线提供。
英文摘要：
We introduce VampNet, a masked acoustic token modeling approach to music synthesis, compression, inpainting, and variation. We use a variable masking schedule during training which allows us to sample coherent music from the model by applying a variety of masking approaches (called prompts) during inference. VampNet is non-autoregressive, leveraging a bidirectional transformer architecture that attends to all tokens in a forward pass. With just 36 sampling passes, VampNet can generate coherent high-fidelity musical waveforms. We show that by prompting VampNet in various ways, we can apply it to tasks like music compression, inpainting, outpainting, continuation, and looping with variation (vamping). Appropriately prompted, VampNet is capable of maintaining style, genre, instrumentation, and other high-level aspects of the music. This flexible prompting capability makes VampNet a powerful music co-creation tool. Code and audio samples are available online.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
