arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.06307cs.SDcs.AIeess.AS
收藏

一种用于和弦条件歌曲生成的端到端方法

An End-to-End Approach for Chord-Conditioned Song Generation

Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Kunlun Skywork Technology Co.(昆仑智作科技有限公司);Peng Cheng Lab(鹏城实验室)

Shuochen Gao, Shun Lei, Fan Zhuo, Hangyu Liu, Feng Liu, Boshi Tang, Qiaochu Huang, Shiyin Kang, Zhiyong Wu

AI总结:

针对歌曲生成中音乐表现与控制不足的问题,提出以和弦为条件、结合动态权重交叉注意力的端到端 CSG 模型,提升生成歌曲的音乐性与控制精度。

摘要:

歌曲生成任务旨在根据给定歌词合成由人声与伴奏组成的音乐。尽管现有方法 Jukebox 已对该任务进行了探索,但其对生成结果的受限控制往往导致音乐表演方面的不足。为缓解这一问题,我们将音乐作曲中的一个重要概念——和弦——引入歌曲生成网络。和弦构成伴奏的基础,并为人声旋律提供相关和声。鉴于自动和弦提取器存在不准确之处,我们设计了一种由动态权重序列增强的鲁棒交叉注意力机制,以将提取出的和弦信息整合到歌曲生成中并减少帧级缺陷,并在此基础上提出了名为 Chord-Conditioned Song Generator(CSG)的新模型。实验证据表明,我们提出的方法在所生成歌曲的音乐表演和控制精度方面优于其他方法。

英文摘要:

The Song Generation task aims to synthesize music composed of vocals and accompaniment from given lyrics. While the existing method, Jukebox, has explored this task, its constrained control over the generations often leads to deficiency in music performance. To mitigate the issue, we introduce an important concept from music composition, namely chords, to song generation networks. Chords form the foundation of accompaniment and provide vocal melody with associated harmony. Given the inaccuracy of automatic chord extractors, we devise a robust cross-attention mechanism augmented with dynamic weight sequence to integrate extracted chord information into song generations and reduce frame-level flaws, and propose a novel model termed Chord-Conditioned Song Generator (CSG) based on it. Experimental evidence demonstrates our proposed method outperforms other approaches in terms of musical performance and control precision of generated songs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑