面向自监督骨架动作识别的跨流对比学习
Cross-Stream Contrastive Learning for Self-Supervised Skeleton-Based Action Recognition
AI总结：
本文提出跨流对比学习框架CSCLR，通过引入流间对比困难样本和正特征变换PFT增强自监督骨架动作表示学习，在多个基准数据集上超越现有方法。
摘要：
随着对比学习的发展，自监督的基于骨架的动作识别得到了快速增长。现有方法依赖于在单一数据流中对3D骨架的增强施加不变性，这仅利用了容易的正样本对，限制了对复杂运动模式的探索能力。本文认为，单流对比的缺陷和必要特征变换的缺失是导致容易正样本的原因，因此提出了一种用于基于骨架的动作表示学习的跨流对比学习框架（CSCLR）。具体而言，所提出的CSCLR不仅利用流内对比对，还引入流间对比对作为困难样本，以形成更好的表示学习。此外，为了进一步挖掘正样本对的潜力并提高自监督表示学习的鲁棒性，我们提出了一种正特征变换（PFT）策略，该策略采用特征级操作来增加正样本对的方差。为验证方法的有效性，我们在三个基准数据集NTU-RGB+D 60、NTU-RGB+D 120和PKU-MMD上进行了广泛实验。实验结果表明，我们提出的CSCLR在多种评估协议上超越了最先进的方法。
英文摘要：
Self-supervised skeleton-based action recognition enjoys a rapid growth along with the development of contrastive learning. The existing methods rely on imposing invariance to augmentations of 3D skeleton within a single data stream, which merely leverages the easy positive pairs and limits the ability to explore the complicated movement patterns. In this paper, we advocate that the defect of single-stream contrast and the lack of necessary feature transformation are responsible for easy positives, and therefore propose a Cross-Stream Contrastive Learning framework for skeleton-based action Representation learning (CSCLR). Specifically, the proposed CSCLR not only utilizes intra-stream contrast pairs, but introduces inter-stream contrast pairs as hard samples to formulate a better representation learning. Besides, to further exploit the potential of positive pairs and increase the robustness of self-supervised representation learning, we propose a Positive Feature Transformation (PFT) strategy which adopts feature-level manipulation to increase the variance of positive pairs. To validate the effectiveness of our method, we conduct extensive experiments on three benchmark datasets NTU-RGB+D 60, NTU-RGB+D 120 and PKU-MMD. Experimental results show that our proposed CSCLR exceeds the state-of-the-art methods on a diverse range of evaluation protocols.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
