MacDiff：基于掩码条件扩散的统一骨架建模
MacDiff: Unified Skeleton Modeling with Masked Conditional Diffusion
AI总结：
针对现有骨架自监督学习方法的缺陷，提出基于掩码条件扩散的统一骨架建模框架MacDiff，兼具表征学习与生成能力，在基准上达SOTA，还可用于数据增强提升少样本微调性能。
摘要：
自监督学习已被证明在基于骨架的人体动作理解中有效。然而，以往的研究要么依赖存在假阴性问题的对比学习，要么基于重建方法学习过多无关的低层线索，导致下游任务的表征能力有限。近年来，生成式学习取得了巨大进展，它本质上是一项具有挑战性但意义重大的前置任务，用于建模数据的通用底层分布。但生成模型的表征学习能力尚未得到充分探索，尤其是针对具有空间稀疏性和时间冗余性的骨架数据。为此，我们提出掩码条件扩散（MacDiff）作为人体骨架建模的统一框架，首次将扩散模型用作高效的骨架表征学习器。具体而言，我们训练一个以语义编码器提取的表征为条件的扩散解码器，对编码器输入施加随机掩码以引入信息瓶颈并去除骨架的冗余信息。此外，我们从理论上证明，我们的生成目标包含了对齐掩码视图与噪声视图的对比学习目标，同时还促使表征对噪声视图进行补充，从而带来更好的泛化性能。MacDiff在表征学习基准上取得了当前最优性能，同时保持了生成任务的能力。此外，我们利用扩散模型进行数据增强，在标注数据稀缺的场景下显著提升了微调性能。我们的项目可在https://lehongwu.github.io/ECCV24MacDiff/ 获取。
英文摘要：
Self-supervised learning has proved effective for skeleton-based human action understanding. However, previous works either rely on contrastive learning that suffers false negative problems or are based on reconstruction that learns too much unessential low-level clues, leading to limited representations for downstream tasks. Recently, great advances have been made in generative learning, which is naturally a challenging yet meaningful pretext task to model the general underlying data distributions. However, the representation learning capacity of generative models is under-explored, especially for the skeletons with spacial sparsity and temporal redundancy. To this end, we propose Masked Conditional Diffusion (MacDiff) as a unified framework for human skeleton modeling. For the first time, we leverage diffusion models as effective skeleton representation learners. Specifically, we train a diffusion decoder conditioned on the representations extracted by a semantic encoder. Random masking is applied to encoder inputs to introduce a information bottleneck and remove redundancy of skeletons. Furthermore, we theoretically demonstrate that our generative objective involves the contrastive learning objective which aligns the masked and noisy views. Meanwhile, it also enforces the representation to complement for the noisy view, leading to better generalization performance. MacDiff achieves state-of-the-art performance on representation learning benchmarks while maintaining the competence for generative tasks. Moreover, we leverage the diffusion model for data augmentation, significantly enhancing the fine-tuning performance in scenarios with scarce labeled data. Our project is available at https://lehongwu.github.io/ECCV24MacDiff/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
