TokenDance: 基于双向Mamba的token-to-token音乐-舞蹈生成
TokenDance: Token-to-Token Music-to-Dance Generation with Bidirectional Mamba
AI总结：
本文提出TokenDance框架，通过双模态token化和高效token级生成，解决现有3D舞蹈数据集覆盖有限的问题，提升音乐-舞蹈生成的质量与效率。
摘要：
音乐-舞蹈生成在虚拟现实、舞蹈教育和数字人物动画中有广泛应用。然而，现有3D舞蹈数据集的有限覆盖范围限制了当前模型只能处理有限的音乐风格和舞蹈模式，导致生成的舞蹈在现实音乐中表现不佳。为此，我们提出TokenDance，一种两阶段音乐-舞蹈生成框架，通过双模态token化和高效token级生成，明确解决这一限制。第一阶段中，舞蹈和音乐均通过有限标量量化进行离散化，其中舞蹈动作分解为上半身和下半身组件，并受运动-动力学约束；音乐则分解为语义和听觉特征，并通过专用代码本捕捉舞蹈特定结构。第二阶段引入基于双向Mamba的本地-全局-本地token-to-token生成器，实现连贯的运动合成、强音乐-舞蹈对齐和高效的非自回归推理。大量实验表明，TokenDance在生成质量和推理速度方面均达到当前最佳水平，凸显其在现实音乐-舞蹈应用中的有效性与实用性。
英文摘要：
Music-to-dance generation has broad applications in virtual reality, dance education, and digital character animation. However, the limited coverage of existing 3D dance datasets confines current models to a narrow subset of music styles and choreographic patterns, resulting in poor generalization to real-world music. Consequently, generated dances often become overly simplistic and repetitive, substantially degrading expressiveness and realism. To tackle this problem, we present TokenDance, a two-stage music-to-dance generation framework that explicitly addresses this limitation through dual-modality tokenization and efficient token-level generation. In the first stage, we discretize both dance and music using Finite Scalar Quantization, where dance motions are factorized into upper and lower-body components with kinematic-dynamic constraints, and music is decomposed into semantic and acoustic features with dedicated codebooks to capture choreography-specific structures. In the second stage, we introduce a Local-Global-Local token-to-token generator built on a Bidirectional Mamba backbone, enabling coherent motion synthesis, strong music-dance alignment, and efficient non-autoregressive inference. Extensive experiments demonstrate that TokenDance achieves overall state-of-the-art (SOTA) performance in both generation quality and inference speed, highlighting its effectiveness and practical value for real-world music-to-dance applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
