面向自监督人体动作识别的跨模型跨流学习
Cross-Model Cross-Stream Learning for Self-Supervised Human Action Recognition
AI总结：
本文提出跨模型跨流（CMCS）框架，结合跨模型对抗学习与跨流协作学习，以SkeletonBYOL为基线，提升自监督骨架动作识别的多流特征判别性与聚合效果，并在多个数据集上取得优于现有方法的结果。
摘要：
考虑到实例级判别能力，包括MoCo和SimCLR在内的对比学习方法已从原始的图像表示学习任务迁移到自监督骨架动作识别任务中。这些方法通常使用多个数据流（即关节流、运动流和骨骼流）进行集成学习，然而如何在单一流内构建具有判别性的特征空间，并有效聚合来自多个流的信息，仍然是一个尚未解决的问题。为此，本文首先应用一种名为BYOL的新型对比学习方法从骨架数据中学习，并将SkeletonBYOL构建为一个简单而有效的自监督骨架动作识别基线。受SkeletonBYOL启发，本文进一步提出了跨模型跨流（CMCS）框架。该框架结合了跨模型对抗学习（CMAL）和跨流协作学习（CSCL）。具体而言，CMAL通过跨模型对抗损失学习单流表示，以获得更具判别性的特征。为了聚合并交互多流信息，CSCL通过生成集成学习的相似性伪标签作为监督，并指导各个流的特征生成。在三个数据集上的大量实验验证了CMAL和CSCL之间的互补特性，并验证了所提出的方法在使用多种评估协议时能够取得优于最先进方法的结果。
英文摘要：
Considering the instance-level discriminative ability, contrastive learning methods, including MoCo and SimCLR, have been adapted from the original image representation learning task to solve the self-supervised skeleton-based action recognition task. These methods usually use multiple data streams (i.e., joint, motion, and bone) for ensemble learning, meanwhile, how to construct a discriminative feature space within a single stream and effectively aggregate the information from multiple streams remains an open problem. To this end, this paper first applies a new contrastive learning method called BYOL to learn from skeleton data, and then formulate SkeletonBYOL as a simple yet effective baseline for self-supervised skeleton-based action recognition. Inspired by SkeletonBYOL, this paper further presents a Cross-Model and Cross-Stream (CMCS) framework. This framework combines Cross-Model Adversarial Learning (CMAL) and Cross-Stream Collaborative Learning (CSCL). Specifically, CMAL learns single-stream representation by cross-model adversarial loss to obtain more discriminative features. To aggregate and interact with multi-stream information, CSCL is designed by generating similarity pseudo label of ensemble learning as supervision and guiding feature generation for individual streams. Extensive experiments on three datasets verify the complementary properties between CMAL and CSCL and also verify that the proposed method can achieve better results than state-of-the-art methods using various evaluation protocols.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
