基于对比学习的双流联合匹配方法在少样本动作识别中的应用
Two-stream joint matching method based on contrastive learning for few-shot action recognition
AI总结：
本文提出基于对比学习的双流联合匹配方法TSJM，通过多模态对比学习模块和联合匹配模块解决少样本动作识别中的多模态信息利用不足及视频匹配问题。
摘要：
尽管基于度量学习范式的少样本动作识别取得了显著成功，但它未能解决以下问题：（1）动作关系建模不充分以及多模态信息利用不足；（2）处理不同长度和速度的视频匹配问题，以及视频子动作错位的视频匹配问题的挑战。为了解决这些问题，我们提出了一种基于对比学习的双流联合匹配方法（TSJM），该方法包含两个模块：多模态对比学习模块（MCL）和联合匹配模块（JMM）。MCL的目标是深入探究模态间的互信息关系，从而充分提取模态信息以增强动作关系的建模。JMM旨在同时解决上述视频匹配问题。所提方法的有效性在两个广泛使用的少样本动作识别数据集，即SSv2和Kinetics上进行了评估。还进行了全面的消融实验以证实我们提出方法的有效性。
英文摘要：
Although few-shot action recognition based on metric learning paradigm has achieved significant success, it fails to address the following issues: (1) inadequate action relation modeling and underutilization of multi-modal information; (2) challenges in handling video matching problems with different lengths and speeds, and video matching problems with misalignment of video sub-actions. To address these issues, we propose a Two-Stream Joint Matching method based on contrastive learning (TSJM), which consists of two modules: Multi-modal Contrastive Learning Module (MCL) and Joint Matching Module (JMM). The objective of the MCL is to extensively investigate the inter-modal mutual information relationships, thereby thoroughly extracting modal information to enhance the modeling of action relationships. The JMM aims to simultaneously address the aforementioned video matching problems. The effectiveness of the proposed method is evaluated on two widely used few shot action recognition datasets, namely, SSv2 and Kinetics. Comprehensive ablation experiments are also conducted to substantiate the efficacy of our proposed approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
