近年来,音乐生成系统的质量不断提升,公众对其关注度也日益增长,这推动了针对各类系统控制方式的研究。我们提出了一种利用序列模型控制音乐生成中惊奇度的新方法。为实现这一目标,我们定义了一种名为瞬时信息含量(Instantaneous Information Content,IIC)的指标。IIC可作为感知音乐惊奇度的代理函数(由概率模型估算得出),且能在音乐作品的任意节点计算,这使得即便音乐事件以不规则时间间隔发生,也能对不同音乐内容的惊奇度进行比较。我们使用束搜索生成IIC曲线与给定目标IIC高度近似的音乐素材。实验表明,IIC与和声复杂度、节奏复杂度以及音符密度存在相关性,且这种相关性会随着估算IIC所用音乐上下文长度的增加而减弱。最后,我们开展了定性用户研究,测试人类听众是否能够识别出生成对应音乐素材时用作目标的IIC曲线。我们在https://github.com/muthissar/iic上提供了用于生成IIC插值和IIC可视化的代码。
英文摘要:
In recent years, the quality and public interest in music generation systems have grown, encouraging research into various ways to control these systems. We propose a novel method for controlling surprisal in music generation using sequence models. To achieve this goal, we define a metric called Instantaneous Information Content (IIC). The IIC serves as a proxy function for the perceived musical surprisal (as estimated from a probabilistic model) and can be calculated at any point within a music piece. This enables the comparison of surprisal across different musical content even if the musical events occur in irregular time intervals. We use beam search to generate musical material whose IIC curve closely approximates a given target IIC. We experimentally show that the IIC correlates with harmonic and rhythmic complexity and note density. The correlation decreases with the length of the musical context used for estimating the IIC. Finally, we conduct a qualitative user study to test if human listeners can identify the IIC curves that have been used as targets when generating the respective musical material. We provide code for creating IIC interpolations and IIC visualizations on https://github.com/muthissar/iic.
EMelodyGen: Emotion-Conditioned Melody Generation in ABC Notation with the Musical Feature Template
Department of Music AI and Information Technology, Central Conservatory of Music(中央音乐学院音乐人工智能与信息科技系);School of Computer Science and Technology, Fudan University(复旦大学计算机科学与技术学院);Shanghai Key Laboratory of Intelligent Information Processing, Fudan University(复旦大学智能信息处理上海重点实验室)
MusicLayout: Explicit Structural Planning for Controllable Text-to-Music Generation
College of Artificial Intelligence, Zhejiang University(浙江大学人工智能学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Innovation Center of Yangtze River Delta, Zhejiang University(浙江大学长三角创新中心);The Chinese University of Hong Kong(香港中文大学);Shandong University(山东大学);Chu Kochen Honors College, Zhejiang University(浙江大学竺可桢学院);Ant Group(蚂蚁集团)