Dancing to music is an instinctive move by humans. Learning to model the music-to-dance generation process is, however, a challenging problem. It requires significant efforts to measure the correlation between music and dance as one needs to simultaneously consider multiple aspects, such as style and beat of both music and dance. Additionally, dance is inherently multimodal and various following movements of a pose at any moment are equally likely. In this paper, we propose a synthesis-by-analysis learning framework to generate dance from music. In the analysis phase, we decompose a dance into a series of basic dance units, through which the model learns how to move. In the synthesis phase, the model learns how to compose a dance by organizing multiple basic dancing movements seamlessly according to the input music. Experimental qualitative and quantitative results demonstrate that the proposed method can synthesize realistic, diverse,style-consistent, and beat-matching dances from music.
Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机科学学院，北京大学）；UCAS-Terminus AI Lab, University of Chinese Academy of Sciences（中国科学院大学Terminus AI实验室，中国科学院大学）；School of Intelligent Science and Technology, Nanjing University（智能科学与技术学院，南京大学）；Department of Information Technology and Electrical Engineering, ETH Zurich（信息科技与电气工程系，苏黎世联邦理工学院）；Department of Electrical Engineering, KU Leuven（电气工程系，鲁汶大学）；INSAIT, Sofia Un.（INSAIT，索菲亚大学）；Department of Information Engineering and Computer Science (DISI), University of Trento（信息工程与计算机科学系（DISI），特伦托大学）
DanceCamera3D: 3D Camera Movement Synthesis with Music and Dance
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology (BNRist)（北京信息科学与技术国家研究中心）；ByteDance（字节跳动）；University of Rochester（罗切斯特大学）
针对音乐与舞蹈的相机运动合成缺乏成对数据的挑战，提出新多模态数据集 DCM 及基于 transformer 的扩散模型 DanceCamera3D，结合 body attention loss 与条件分离策略，并设计新评估指标验证了有效性。
Exploring Multi-Modal Control in Music-Driven Dance Generation
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Peng Cheng Laboratory（鹏城实验室）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；School of Electronics Engineering and Computer Science, Peking University（北京大学电子工程与计算机科学学院）；National Institute of Health Data Science, Peking University（北京大学健康数据科学研究所）；State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）
PAMD: Plausibility-Aware Motion Diffusion Model for Long Dance Generation
School of Computer Science and Engineering, Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, Southeast University（计算机科学与工程学院，新一代人工智能技术及交叉应用重点实验室，教育部，东南大学）；State Key Laboratory of Media Convergence and Communication, Communication University of China（媒体融合与传播国家重点实验室，中国传媒大学）；School of Computer Science and Software Engineering, National Engineering Laboratory for Big Data System Computing Technology, Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen University（计算机科学与软件工程学院，大数据系统计算技术国家工程实验室，广东智能信息处理重点实验室，深圳大学）；School of Computer Science and Engineering, Nanjing University of Science and Technology（计算机科学与工程学院，南京理工大学）
Tempo as the Stable Cue: Hierarchical Mixture of Tempo and Beat Experts for Music to 3D Dance Generation
School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；Hohai university（河海大学）；School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Institute for Infocomm Research (I 2 R), A*STAR（新加坡科技研究局信息与通信研究所）
GCDance: Genre-Controlled Music-Driven 3D Full Body Dance Generation
School of Computer Science and Electronic Engineering, University of Surrey（萨里大学计算机科学与电子工程学院）；Department of Music and Media, University of Surrey（萨里大学音乐与媒体系）；Department of Informatics, University of Oslo（奥斯陆大学信息学系）；Centre for Vision, Speech and Signal Processing, University of Surrey（萨里大学视觉、语音与信号处理中心）；School of Artificial Intelligence and Computer Science, Jiangnan University（江南大学人工智能与计算机科学学院）
Align Your Rhythm: Generating Highly Aligned Dance Poses with Gating-Enhanced Rhythm-Aware Feature Representation
College of Computer Science and Technology, Harbin Engineering University（哈尔滨工程大学计算机科学与技术学院）；Shanghai Academy of Artificial Intelligence for Science（上海人工智能科学研究院）；School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）；State Key Laboratory of Space-Ground Integrated Information Technology（空间地面一体化信息技术创新实验室）
TCDiff++: An End-to-end Trajectory-Controllable Diffusion Model for Harmonious Music-Driven Group Choreography
PCA Lab, Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China（教育部高维信息智能感知与系统重点实验室，南京理工大学计算机科学与工程学院）；Shenzhen International Graduate School, Tsinghua University, Shenzhen, China（清华大学深圳国际研究生院）；Nanjing University, Suzhou, China（南京大学）
QEAN: Quaternion-Enhanced Attention Network for Visual Dance Generation
Guangdong University of Technology（广东工业大学）；Huizhou University（惠州学院）；Guangdong Mechanical and Electrical College（广东机电职业技术学院）；University of Western Australia（西澳大学）
