arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.09891cs.SDcs.LGcs.MM
收藏

Stemphonic:一次生成灵活多音部音乐创作

Stemphonic: All-at-once Flexible Multi-stem Music Generation

MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);Adobe Research(Adobe研究院)

Shih-Lun Wu, Ge Zhu, Juan-Pablo Caceres, Cheng-Zhi Anna Huang, Nicholas J. Bryan

AI总结:

Stemphonic通过一次生成灵活多音部音乐创作,提升音乐生成效率和质量。

摘要:

音乐音部生成任务,即生成音乐同步且隔离的乐器音频片段,相比传统文本到音乐模型,提供了更大的用户控制和更好的音乐家工作流程对齐潜力。然而,现有音部生成方法要么依赖于固定架构,输出预定义的音部集合,要么一次只生成一个音部,导致推理缓慢尽管音部组合灵活。我们提出了Stemphonic,一种基于扩散/流的框架,克服了这一权衡,一次推理过程中生成可变的同步音部集合。在训练过程中,我们将每个音部视为一个批量元素,将同步的音部分组到一个批量中,并为每个组应用共享的噪声潜在特征。在推理时,我们使用共享的初始噪声潜在特征和音部特定的文本输入,一次生成同步的多音部输出。我们进一步扩展我们的方法,使能够一次条件生成多音部和音部级活动控制,以使用户能够迭代生成和编排混合的时序层次。我们在多个开源音部评估集上基准测试我们的结果,并显示Stemphonic生成的输出质量更高,同时将完整混合生成过程的速度加快了25至50%。演示地址:https://stemphonic-demo.vercel.app。

英文摘要:

Music stem generation, the task of producing musically-synchronized and isolated instrument audio clips, offers the potential of greater user control and better alignment with musician workflows compared to conventional text-to-music models. Existing stem generation approaches, however, either rely on fixed architectures that output a predefined set of stems in parallel, or generate only one stem at a time, resulting in slow inference despite flexibility in stem combination. We propose Stemphonic, a diffusion-/flow-based framework that overcomes this trade-off and generates a variable set of synchronized stems in one inference pass. During training, we treat each stem as a batch element, group synchronized stems in a batch, and apply a shared noise latent to each group. At inference-time, we use a shared initial noise latent and stem-specific text inputs to generate synchronized multi-stem outputs in one pass. We further expand our approach to enable one-pass conditional multi-stem generation and stem-wise activity controls to empower users to iteratively generate and orchestrate the temporal layering of a mix. We benchmark our results on multiple open-source stem evaluation sets and show that Stemphonic produces higher-quality outputs while accelerating the full mix generation process by 25 to 50%. Demos at: https://stemphonic-demo.vercel.app.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑