基于跨模态 Transformer 的舞蹈风格迁移
Dance Style Transfer with Cross-modal Transformer
AI总结：
提出 CycleDance 舞蹈风格迁移系统，通过扩展 CycleGAN 架构并结合多模态 Transformer 编码器与课程学习，在保留动作上下文的同时实现跨风格舞蹈迁移，显著优于基线方法。
摘要：
我们提出了 CycleDance，一种舞蹈风格迁移系统，能将一种舞蹈风格的现有动作片段转换为另一种舞蹈风格的动作片段，同时尽量保留舞蹈的动作上下文。我们的方法扩展现有的 CycleGAN 架构以对音频序列进行建模，并集成多模态 Transformer 编码器来考虑音乐上下文。我们采用基于序列长度的课程学习来稳定训练。我们的方法捕捉了动作帧之间丰富且长期的内部关系，这是动作迁移和合成工作中常见的挑战。我们进一步引入了新的指标，用于在舞蹈动作的背景下衡量迁移强度和内容保留程度。我们进行了广泛的消融研究，以及一项包含 30 名具有 5 年或以上舞蹈经验参与者的人类研究。结果表明，CycleDance 生成了具有目标风格的逼真动作，在自然度、迁移强度和内容保留方面显著优于基线 CycleGAN。
英文摘要：
We present CycleDance, a dance style transfer system to transform an existing motion clip in one dance style to a motion clip in another dance style while attempting to preserve motion context of the dance. Our method extends an existing CycleGAN architecture for modeling audio sequences and integrates multimodal transformer encoders to account for music context. We adopt sequence length-based curriculum learning to stabilize training. Our approach captures rich and long-term intra-relations between motion frames, which is a common challenge in motion transfer and synthesis work. We further introduce new metrics for gauging transfer strength and content preservation in the context of dance movements. We perform an extensive ablation study as well as a human study including 30 participants with 5 or more years of dance experience. The results demonstrate that CycleDance generates realistic movements with the target style, significantly outperforming the baseline CycleGAN on naturalness, transfer strength, and content preservation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
