arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.04865cs.SDcs.MMeess.AS

TEAdapter:为可控文本到音乐生成提供丰富指导

TEAdapter: Supply abundant guidance for controllable text-to-music generation

East China Normal University(华东师范大学)

Jialing Zou, Jiahao Mei, Xudong Nan, Jinghua Li, Daoguo Dong, Liang He

AI总结:

针对文本到音乐生成中细粒度多模态控制难题,本文提出轻量插件 TEAdapter,支持全局、元素与结构层面的多样化精确控制,并可迁移至任意扩散模型架构。

摘要:

尽管当前文本引导的音乐生成技术能够应对简单的创作场景,但随着用户需求变得更加复杂,实现对各个文本模态条件的细粒度控制仍然具有挑战性。因此,我们提出了 TEAcher Adapter(TEAdapter),这是一个紧凑的插件,旨在利用用户提供的多样化控制信息来引导生成过程。此外,我们通过利用在具有不同结构功能的数据上训练的 TEAdapter 控制组,探索了扩展音乐的可控生成。总体而言,我们考虑了全局、元素和结构层面的控制。实验结果表明,所提出的 TEAdapter 能够实现多种精确控制,并确保高质量的音乐生成。我们的模块还很轻量,并且可迁移到任何扩散模型架构。可用的代码和演示很快将在 https://github.com/Ashley1101/TEAdapter 上提供。

英文摘要:

Although current text-guided music generation technology can cope with simple creative scenarios, achieving fine-grained control over individual text-modality conditions remains challenging as user demands become more intricate. Accordingly, we introduce the TEAcher Adapter (TEAdapter), a compact plugin designed to guide the generation process with diverse control information provided by users. In addition, we explore the controllable generation of extended music by leveraging TEAdapter control groups trained on data of distinct structural functionalities. In general, we consider controls over global, elemental, and structural levels. Experimental results demonstrate that the proposed TEAdapter enables multiple precise controls and ensures high-quality music generation. Our module is also lightweight and transferable to any diffusion model architecture. Available code and demos will be found soon at https://github.com/Ashley1101/TEAdapter.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑