CustomDance：基于以人为中心的粗到细交互式控制的定制化3D舞蹈生成
CustomDance: Customized 3D Dance Generation with Coarse-to-Fine Human-Centered Interactive Control
- University of Texas at Dallas（达拉斯大学）
- MalouTech Inc（MalouTech公司）
- University at Albany（奥尔巴尼大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
CustomDance是基于粗到细交互式控制的定制化3D舞蹈生成系统，通过三阶段流程实现AI辅助编舞，在定量和定性比较中优于基线，为用户提供全面控制。
AI中文摘要：
随着AI生成内容（AIGC）和先进3D人体表示技术的兴起，生成3D舞蹈动作已成为令人兴奋的研究领域。尽管取得了重大进展，当前方法往往无法对用户的各种多模态输入（如音乐或期望动作的特定描述）提供全面且明确的控制。因此，生成的动作可能在统计上合理且技术上正确，但往往缺乏深度、表现力，且与用户的创作愿景不一致。为解决该问题，我们提出CustomDance，这是一个用于定制化3D舞蹈生成的粗到细交互式系统。受专业编舞家工作流程的启发，CustomDance通过三个相互关联的阶段引入了AI辅助编舞的新范式：首先，多模态大语言模型（MLLM）分析音乐和高级文本提示，以识别作品的关键时间锚点和创作线索；其次，针对每个锚点，多模态检索器基于局部音乐和文本从舞蹈库中推荐高质量动作片段，为用户提供具体且可预测的选项；最后，定制化音乐条件扩散修复器无缝连接所选片段，支持用户对最终作品进行迭代引导式优化，同时辅以动作动力学可视化。我们的评估表明，CustomDance不仅凸显了AI辅助编舞范式的重大创作实用性和赋能潜力，还在定量和定性比较中优于竞争性基线。
英文摘要：
With the rise of AI-generated content (AIGC) and advanced techniques for 3D human representation, the task of generating 3D dance movements has become an exciting area of research. Despite significant advancements, current methods often fail to provide comprehensive and distinct control over various multimodal inputs from users, such as music or specific descriptions of desired movements. As a result, the generated motions may be statistically plausible and technically correct, but they often lack depth, expressiveness, and alignment with the user's creative vision. To address this issue, we present CustomDance, a coarse-to-fine interactive system designed for customized 3D dance generation. Inspired by the workflows of expert choreographers, CustomDance introduces a novel paradigm to AI-assisted choreography through three interconnected stages. First, a multimodal Large Language Model (MLLM) analyzes the music and a high-level text prompt to identify key temporal anchors and creative cues for the piece. Next, for each anchor, a multimodal retriever suggests high-quality motion clips from a dance library based on local music and text, empowering the user with concrete and predictable options. Finally, a custom music-conditioned diffusion in-painter seamlessly connects the selected phrases, allowing for iterative, user-guided refinement of the final composition, supported by visualizations of motion dynamics. Our evaluations demonstrate that CustomDance not only highlights the significant creative utility and empowering potential of our AI-assisted choreography paradigm, but also outperforms competitive baselines across quantitative and qualitative comparisons. Project page: https://github.com/XulongT/CustomDance
