CMD：基于跨模态互蒸馏的自监督三维动作表示学习
CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation
AI总结：
本文提出跨模态互蒸馏框架CMD，通过近邻相似性分布建模和非对称教师-学生配置实现骨架模态间双向知识蒸馏，在多个三维动作识别数据集上超越现有自监督方法并刷新纪录。
摘要：
在三维动作识别中，不同骨架模态之间存在丰富的互补信息。然而，如何建模并利用这些信息，对于自监督三维动作表示学习而言仍是一个具有挑战性的问题。本文将跨模态交互形式化为一个双向知识蒸馏问题。与将固定且预训练好的教师知识迁移给学生的经典蒸馏方案不同，本文中的知识在模态之间持续更新并双向蒸馏。为此，我们提出了一种新的跨模态互蒸馏（CMD）框架，其设计如下：一方面，引入近邻相似性分布来建模每个模态中学到的知识，其中关系信息天然适用于对比学习框架；另一方面，教师和学生采用非对称配置，以稳定蒸馏过程并在模态之间传递高置信度信息。通过推导，我们发现以往工作中的跨模态正样本挖掘可以视为本文CMD的一个退化版本。我们在NTU RGB+D 60、NTU RGB+D 120和PKU-MMD II数据集上进行了大量实验。我们的方法优于现有的自监督方法，并创造了一系列新纪录。代码可在 https://github.com/maoyunyao/CMD 获取。
英文摘要：
In 3D action recognition, there exists rich complementary information between skeleton modalities. Nevertheless, how to model and utilize this information remains a challenging problem for self-supervised 3D action representation learning. In this work, we formulate the cross-modal interaction as a bidirectional knowledge distillation problem. Different from classic distillation solutions that transfer the knowledge of a fixed and pre-trained teacher to the student, in this work, the knowledge is continuously updated and bidirectionally distilled between modalities. To this end, we propose a new Cross-modal Mutual Distillation (CMD) framework with the following designs. On the one hand, the neighboring similarity distribution is introduced to model the knowledge learned in each modality, where the relational information is naturally suitable for the contrastive frameworks. On the other hand, asymmetrical configurations are used for teacher and student to stabilize the distillation process and to transfer high-confidence information between modalities. By derivation, we find that the cross-modal positive mining in previous works can be regarded as a degenerated version of our CMD. We perform extensive experiments on NTU RGB+D 60, NTU RGB+D 120, and PKU-MMD II datasets. Our approach outperforms existing self-supervised methods and sets a series of new records. The code is available at: https://github.com/maoyunyao/CMD
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
