增强的细粒度运动扩散用于文本驱动的人体运动合成
Enhanced Fine-grained Motion Diffusion for Text-driven Human Motion Synthesis
AI总结：
本文提出DiffKFC，一种带关键帧协作的条件扩散模型，通过语义级粗引导和姿态级细粒度关键帧控制，实现文本驱动的人体运动合成，提升语义保真度并满足动画师需求。
摘要：
文本驱动的运动合成技术的出现为动画师提供了高效创作的巨大潜力。然而，在大多数情况下，文本表达仅包含一般性和定性的运动描述，缺乏细致的描绘和足够的强度，导致合成运动要么（a）语义上符合但无法控制特定姿态细节，要么（b）甚至偏离所提供的描述，给动画师带来不期望的结果。在本文中，我们提出了DiffKFC，一种带有关键帧协作的文本驱动运动合成条件扩散模型，能够通过协作且高效的双层控制实现逼真生成：语义层面的粗略引导，仅需少量关键帧即可直接、细粒度地描绘到身体姿态层面。与现有的无需训练即可纳入条件的推理编辑扩散模型不同，我们的条件扩散模型经过显式训练，能够充分利用文本、关键帧和扩散目标帧之间的相关性。为了保留离散且稀疏关键帧的控制能力，我们定制了膨胀掩码注意力模块，其中仅部分有效令牌参与从局部到全局的注意力，由膨胀关键帧掩码指示。此外，我们开发了一种简单而有效的平滑先验，在推理时将生成帧引导向无缝的关键帧过渡。大量实验表明，我们的模型不仅在语义保真度方面达到了最先进的性能，更重要的是，能够通过细粒度引导满足动画师的需求，而无需繁琐的劳动。
英文摘要：
The emergence of text-driven motion synthesis technique provides animators with great potential to create efficiently. However, in most cases, textual expressions only contain general and qualitative motion descriptions, while lack fine depiction and sufficient intensity, leading to the synthesized motions that either (a) semantically compliant but uncontrollable over specific pose details, or (b) even deviates from the provided descriptions, bringing animators with undesired cases. In this paper, we propose DiffKFC, a conditional diffusion model for text-driven motion synthesis with KeyFrames Collaborated, enabling realistic generation with collaborative and efficient dual-level control: coarse guidance at semantic level, with only few keyframes for direct and fine-grained depiction down to body posture level. Unlike existing inference-editing diffusion models that incorporate conditions without training, our conditional diffusion model is explicitly trained and can fully exploit correlations among texts, keyframes and the diffused target frames. To preserve the control capability of discrete and sparse keyframes, we customize dilated mask attention modules where only partial valid tokens participate in local-to-global attention, indicated by the dilated keyframe mask. Additionally, we develop a simple yet effective smoothness prior, which steers the generated frames towards seamless keyframe transitions at inference. Extensive experiments show that our model not only achieves state-of-the-art performance in terms of semantic fidelity, but more importantly, is able to satisfy animator requirements through fine-grained guidance without tedious labor.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
