扩散模型已经 emerged 为在人类运动合成中广泛使用和成功的方法。任务导向的扩散模型在动作到运动、文本到运动和音频到运动应用中显著推进了发展。在本文中,我们调查了运动表示和损失函数的基本问题,并在受控研究中列举了生成运动扩散模型工作流程中各种决策的影响。为了回答这些问题,我们基于代理运动扩散模型(MDM)进行了实证研究。我们应用 v loss 作为 MDM(vMDM)的预测目标,其中 v 是运动数据和噪声的加权和。我们的目标是增强对潜在数据分布的理解,并为改进条件运动扩散模型提供基础。首先,我们评估了文献中六种常见的运动表示,并在质量和多样性指标上比较其性能。其次,我们比较了不同配置下的训练时间,以揭示如何加快运动扩散模型的训练过程。最后,我们还对大规模运动数据集进行了评估分析。我们的实验结果表明,在不同数据集上运动表示之间存在明显的性能差异。我们的结果也展示了不同配置对模型训练的影响,并表明这些决策对运动扩散模型结果的重要性与有效性。
英文摘要:
Diffusion models have emerged as a widely utilized and successful methodology in human motion synthesis. Task-oriented diffusion models have significantly advanced action-to-motion, text-to-motion, and audio-to-motion applications. In this paper, we investigate fundamental questions regarding motion representations and loss functions in a controlled study, and we enumerate the impacts of various decisions in the workflow of the generative motion diffusion model. To answer these questions, we conduct empirical studies based on a proxy motion diffusion model (MDM). We apply v loss as the prediction objective on MDM (vMDM), where v is the weighted sum of motion data and noise. We aim to enhance the understanding of latent data distributions and provide a foundation for improving the state of conditional motion diffusion models. First, we evaluate the six common motion representations in the literature and compare their performance in terms of quality and diversity metrics. Second, we compare the training time under various configurations to shed light on how to speed up the training process of motion diffusion models. Finally, we also conduct evaluation analysis on a large motion dataset. The results of our experiments indicate clear performance differences across motion representations in diverse datasets. Our results also demonstrate the impacts of distinct configurations on model training and suggest the importance and effectiveness of these decisions on the outcomes of motion diffusion models.
Learning Normal Diffusion Dynamics for Backdoor Defense in Text-to-Image Models
Geely(吉利);Hunan University(湖南大学);East China Normal University(华东师范大学);University of Adelaide(阿德莱德大学);The Hong Kong University of Science and Technology(香港科技大学);China University of Petroleum (East China)(中国石油大学(华东))
MobileVidFactory: Automatic Diffusion-Based Social Media Video Generation for Mobile Devices from Text
University of Electronic Science and Technology of China(电子科技大学);Microsoft Research(微软研究院);University of Rochester(罗切斯特大学);National Taiwan University(国立台湾大学)
EmoAttack: Emotion-to-Image Diffusion Models for Emotional Backdoor Generation
School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件工程学院);Tongji University(同济大学);College of Computer Science, Nankai University(南开大学计算机科学学院)
UFID: A Unified Framework for Input-level Backdoor Detection on Diffusion Models
University of Virginia(弗吉尼亚大学);Biocomplexity Institute, University of Virginia(弗吉尼亚大学生物复杂性研究所);School of Data Science, University of Virginia(弗吉尼亚大学数据科学学院)