用于舞蹈生成的双向自回归扩散模型
Bidirectional Autoregressive Diffusion Model for Dance Generation
AI总结：
针对现有舞蹈生成模型缺乏双向与局部增强的问题，提出带双向编码器、局部解码器且融入节拍信息的BADM模型，在音乐生成舞蹈基准上实现SOTA性能。
摘要：
舞蹈是表达人类情感的有力媒介，但逼真的舞蹈生成仍是一项相当大的挑战。近年来，扩散模型在多个领域展现出卓越的生成能力，凭借其适配的多对多特性，在人体动作生成领域颇具潜力。然而，当前基于扩散的动作生成模型通常直接单向生成完整动作序列，缺乏对动作的局部与双向增强关注。在编排高质量舞蹈动作时，人们不仅需要考虑音乐语境，还需兼顾附近与音乐对齐的舞蹈动作。为真实捕捉人类行为，我们提出用于音乐生成舞蹈的双向自回归扩散模型（Bidirectional Autoregressive Diffusion Model，BADM），该模型构建了双向编码器，确保生成的舞蹈在正向和反向方向上均协调一致。为使生成的舞蹈动作更流畅，我们构建了局部信息解码器用于局部动作增强。所提出的框架能够基于输入条件和附近动作生成新动作，迭代预测单个动作片段并整合所有预测结果。为进一步优化生成舞蹈与节拍的同步性，模型将节拍信息作为输入，以生成更贴合音乐的舞蹈动作。实验结果表明，在主流的音乐生成舞蹈基准测试上，与现有单向方法相比，所提模型取得了当前最优的性能。
英文摘要：
Dance serves as a powerful medium for expressing human emotions, but the lifelike generation of dance is still a considerable challenge. Recently, diffusion models have showcased remarkable generative abilities across various domains. They hold promise for human motion generation due to their adaptable many-to-many nature. Nonetheless, current diffusion-based motion generation models often create entire motion sequences directly and unidirectionally, lacking focus on the motion with local and bidirectional enhancement. When choreographing high-quality dance movements, people need to take into account not only the musical context but also the nearby music-aligned dance motions. To authentically capture human behavior, we propose a Bidirectional Autoregressive Diffusion Model (BADM) for music-to-dance generation, where a bidirectional encoder is built to enforce that the generated dance is harmonious in both the forward and backward directions. To make the generated dance motion smoother, a local information decoder is built for local motion enhancement. The proposed framework is able to generate new motions based on the input conditions and nearby motions, which foresees individual motion slices iteratively and consolidates all predictions. To further refine the synchronicity between the generated dance and the beat, the beat information is incorporated as an input to generate better music-aligned dance movements. Experimental results demonstrate that the proposed model achieves state-of-the-art performance compared to existing unidirectional approaches on the prominent benchmark for music-to-dance generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
