聆听节奏，选择动作：通过扩散与Mamba实现多模态舞蹈生成的自回归方法
Listen to Rhythm, Choose Movements: Autoregressive Multimodal Dance Generation via Diffusion and Mamba with Decoupled Dance Dataset
AI总结：
本文提出LRCM框架，通过解耦舞蹈数据集实现多模态指导的扩散模型，结合Mamba模块实现流畅的长序列自回归生成，展示了在多模态输入和长序列生成中的潜力。
摘要：
生成模型和序列学习的进步显著推动了舞蹈动作生成研究，但当前方法仍存在语义控制粗略和长序列不连贯的问题。本文提出Listen to Rhythm, Choose Movements (LRCM)，一种多模态指导的扩散框架，支持多种输入模态和自回归舞蹈动作生成。我们探索了一种特征解耦范式，将舞蹈数据集推广到Motorica舞蹈数据集，分离动作捕捉数据、音频节奏和专业标注的全局和局部文本描述。我们的扩散架构整合了音频潜在Conformer和文本潜在Cross-Conformer，并结合Motion Temporal Mamba模块（MTMM）以实现平滑、长持续时间的自回归合成。实验结果表明，LRCM在功能能力和定量指标上均表现出色，展示了在多模态输入场景和扩展序列生成中的显著潜力。项目页面可在https://oranduanstudy.github.io/LRCM/上访问。
英文摘要：
Advances in generative models and sequence learning have greatly promoted research in dance motion generation, yet current methods still suffer from coarse semantic control and poor coherence in long sequences. In this work, we present Listen to Rhythm, Choose Movements (LRCM), a multimodal-guided diffusion framework supporting both diverse input modalities and autoregressive dance motion generation. We explore a feature decoupling paradigm for dance datasets and generalize it to the Motorica Dance dataset, separating motion capture data, audio rhythm, and professionally annotated global and local text descriptions. Our diffusion architecture integrates an audio-latent Conformer and a text-latent Cross-Conformer, and incorporates a Motion Temporal Mamba Module (MTMM) to enable smooth, long-duration autoregressive synthesis. Experimental results indicate that LRCM delivers strong performance in both functional capability and quantitative metrics, demonstrating notable potential in multimodal input scenarios and extended sequence generation. The project page is available at https://oranduanstudy.github.io/LRCM/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
