Music accompaniment generation aims to automatically produce instrumental accompaniments that are rhythmically, harmonically, and timbrally coherent with a given vocal input, with broad applications in personalized music creation, arrangement assistance, and music education. Existing approaches, primarily operating in the symbolic domain or relying on single-stage audio generation frameworks, commonly suffer from insufficient high-level semantic structure modeling, limited acoustic detail reconstruction, and weak conditional controllability. To address these limitations, this paper proposes HAFM, a Hierarchical Autoregressive Foundation Model for vocal-conditioned music accompaniment generation. The model employs a dual-rate tokenization strategy in which $50$ Hz HuBERT semantic tokens capture high-level musical structure and $75$ Hz EnCodec acoustic tokens encode fine-grained acoustic content, enabling explicit disentanglement of semantic and acoustic representations. Building on this foundation, a three-stage cascaded generation framework is designed to progressively generate semantic tokens, coarse acoustic tokens, and fine acoustic tokens, refining the accompaniment from global structure to local detail. . Objective evaluation on the MUSDB18 dataset demonstrates that the full three-stage model achieves a Fr{é}chet Audio Distance (FAD) score of 1.71, representing an 18.6% relative improvement over the two-stage baseline (FAD = 2.10). Subjective listening tests show that the generated accompaniments achieve a 51.5% preference rate against ground-truth accompaniments in head-to-head comparisons, and substantially outperform the random baseline in terms of rhythmic alignment, harmonic compatibility, and overall musical coherence. The source code and demo are available at https://github.com/HackerHyper/HAFM.git.
Towards Unified Song Generation and Singing Voice Conversion with Accompaniment Co-Generation
Northwestern Polytechnical University(西北工业大学);Kuaishou Technology(快手科技);Beijing Institute of Technology(北京理工大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Science and Technology of China(中国科学技术大学);Shanghai Jiao Tong University(上海交通大学)
Drop the beat! Freestyler for Accompaniment Conditioned Rapping Voice Generation
Northwestern Polytechnical University(西北工业大学);Microsoft(微软);Shenzhen Research Institute of Big Data(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
SongDriver: Real-time Music Accompaniment Generation without Logical Latency nor Exposure Bias
Zhejiang University(浙江大学);Alibaba-Zhejiang University Joint Institute of Frontier Technologies(阿里巴巴-浙江大学前沿技术联合研究院);Jingchu University of Technology(荆楚理工学院);Shandong University(山东大学)
An Overview of Lead and Accompaniment Separation in Music
Gracenote(Gracenote公司);Inria(法国国家信息与自动化研究所);LIRMM(蒙彼利埃计算机、机器人与微电子实验室);Cork School of Music(科克音乐学院);Cork Institute of Technology(科克理工学院);Northwestern University(西北大学)
Mila, Quebec Artificial Intelligence Institute, Université de Montréal(魁北克人工智能研究所,蒙特利尔大学);MIT - Massachusetts Institute of Technology(麻省理工学院);Canada CIFAR AI Chair(加拿大 CIFAR 人工智能 chair)
A Framework for Automated Pop-song Melody Generation with Piano Accompaniment Arrangement
New York University Shanghai(上海纽约大学);Fudan University(复旦大学)
我们贡献了一个流行歌曲自动化框架的主旋律生成和伴奏安排。该框架反映了人类音乐创作的主要程序,通过统一的策略生成主旋律和钢琴伴奏。具体来说,我们以和弦进行为输入,并提出了三种模型来生成一个结构化的旋律与钢琴伴奏纹理。首先,和声交替模型将原始输入和弦进行变换为经改变的和弦进行,以更好地适应指定的音乐风格。第二,旋律生成模型使用季节性ARMA(自回归移动平均)过程生成伴奏的主旋律和其他声音(旋律线)。第三,旋律整合模式将旋律线(人声)整合在一起作为最终的钢琴伴奏。我们使用主观听力测试评估所提出的框架。实验结果表明,生成的旋律评分明显高于双向LSTM生成的旋律,我们的伴奏编排结果与最先进的商业软件Band in a Box相当。
MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment
Research Center for Information Technology Innovation, Academia Sinica(中央研究院信息技术创新研究中心);Department of Computer Science, National Tsing Hua University(国立清华大学计算机科学系)
Structured Multi-Track Accompaniment Arrangement via Style Prior Modelling
Institute of Data Science, NUS(新加坡国立大学数据科学研究院);Integrative Sciences and Engineering Programme, NUS Graduate School(新加坡国立大学研究生院整合科学与工程项目);Machine Learning Department, MBZUAI(穆罕默德·本·扎耶德人工智能大学机器学习系);Computer Science Department, NYU Shanghai(上海纽约大学计算机科学系);School of Computing, NUS(新加坡国立大学计算机学院)
SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation
Beihang University(北京航空航天大学);Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Harbin Institute of Technology(哈尔滨工业大学);CPII under InnoHK(InnoHK下的CPII)
LaDA-Band: Language Diffusion Models for Vocal-to-Accompaniment Generation
State Key Lab of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences (CAS)(中国科学院计算技术研究所人工智能安全国家重点实验室);Lyra Lab, Tencent Music Entertainment(腾讯音乐娱乐集团Lyra实验室);Pengcheng Laboratory(鹏城实验室)
SAMUeL: Efficient Vocal-Conditioned Music Generation via Soft Alignment Attention and Latent Diffusion
Division of Engineering Science University of Toronto(工程科学系 马尼托瓦大学);Innovative Cognitive Computing Center King Mongkut's University of Technology Thonburi(创新认知计算中心 王后大学技术吞武里)