DanceCamAnimator:基于关键帧的可控3D舞蹈相机合成
DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis
AI总结:
针对舞蹈相机运动合成中的抖动和平滑问题,提出模仿人类动画流程的端到端框架DanceCamAnimator,将其分为关键帧检测、合成与补间预测三阶段,实现了基于关键帧且长度可变的可控3D相机运动合成,在DCM数据集上超越基线。
摘要:
由于舞蹈摄影术中相互矛盾的要求和复杂性,从音乐和舞蹈中合成相机运动极具挑战性。不同于始终连续的人类运动,舞蹈相机运动既包含长度可变的连续序列,也包含模拟多机位切换的突发剧烈变化。然而,在以往的工作中,每一帧相机画面都被同等对待,这导致了抖动以及在后期处理中不可避免的平滑处理。为了解决这些问题,我们提出通过将此任务表述为三阶段过程来整合动画师的舞蹈摄影知识:关键帧检测、关键帧合成和补间函数预测。基于此表述,我们设计了一种新颖的端到端舞蹈相机合成框架DanceCamAnimator,它模仿人类动画制作流程,并展现出强大的基于关键帧且长度可变的可控性。在DCM数据集上的大量实验表明,我们的方法在定量和定性上均超越了以往的基线。代码将发布于https://github.com/Carmenw1203/DanceCamAnimator-Official。
英文摘要:
Synthesizing camera movements from music and dance is highly challenging due to the contradicting requirements and complexities of dance cinematography. Unlike human movements, which are always continuous, dance camera movements involve both continuous sequences of variable lengths and sudden drastic changes to simulate the switching of multiple cameras. However, in previous works, every camera frame is equally treated and this causes jittering and unavoidable smoothing in post-processing. To solve these problems, we propose to integrate animator dance cinematography knowledge by formulating this task as a three-stage process: keyframe detection, keyframe synthesis, and tween function prediction. Following this formulation, we design a novel end-to-end dance camera synthesis framework \textbf{DanceCamAnimator}, which imitates human animation procedures and shows powerful keyframe-based controllability with variable lengths. Extensive experiments on the DCM dataset demonstrate that our method surpasses previous baselines quantitatively and qualitatively. Code will be available at \url{https://github.com/Carmenw1203/DanceCamAnimator-Official}.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。