OpenDance: 基于大规模互联网数据的多模态可控3D舞蹈生成
OpenDance: Multimodal Controllable 3D Dance Generation with Large-scale Internet Data
AI总结：
OpenDance通过构建大规模舞蹈数据集和提出统一的可控生成框架，实现了多模态的3D舞蹈生成，支持音乐、文本、关键点和轨迹等多种条件控制。
摘要：
音乐驱动的3D舞蹈生成具有显著的创造性潜力，但实际应用需要灵活且多模态的控制。作为高度动态且复杂的舞蹈运动，涵盖各种风格和流派，舞蹈生成需要满足超越音乐（例如空间轨迹、关键帧手势或风格描述）的多样化条件。然而，缺乏大规模且丰富标注的数据集严重阻碍了进展。在本文中，我们构建了OpenDanceSet，一个包含超过100小时、涵盖14种流派和147名参与者的广泛人类舞蹈数据集。每个样本都有丰富的注释，以促进稳健的跨模态学习：3D运动、配对音乐、2D关键点、轨迹以及专家注释的文本描述。此外，我们提出了OpenDanceNet，一个统一的掩码建模框架，用于可控的舞蹈生成，包括一个解耦的自动编码器和一个多模态联合预测Transformer。OpenDanceNet支持基于音乐和任意组合的文本、关键点或轨迹的生成。全面的实验表明，我们的工作实现了高保真的合成，具有强多样性和逼真的物理接触，同时还提供了对空间和风格条件的灵活控制。项目页面：https://open-dance.github.io
英文摘要：
Music-driven 3D dance generation offers significant creative potential, yet practical applications demand versatile and multimodal control. As the highly dynamic and complex human motion covering various styles and genres, dance generation requires satisfying diverse conditions beyond just music (e.g., spatial trajectories, keyframe gestures, or style descriptions). However, the absence of a large-scale and richly annotated dataset severely hinders progress. In this paper, we build OpenDanceSet, an extensive human dance dataset comprising over 100 hours across 14 genres and 147 subjects. Each sample has rich annotations to facilitate robust cross-modal learning: 3D motion, paired music, 2D keypoints, trajectories, and expert-annotated text descriptions. Furthermore, we propose OpenDanceNet, a unified masked modeling framework for controllable dance generation, including a disentangled auto-encoder and a multimodal joint-prediction Transformer. OpenDanceNet supports generation conditioned on music and arbitrary combinations of text, keypoints, or trajectories. Comprehensive experiments demonstrate that our work achieves high-fidelity synthesis with strong diversity and realistic physical contacts, while also offering flexible control over spatial and stylistic conditions. Project Page: https://open-dance.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
