面向连续依赖数据的自监督对比学习理论框架
A theoretical framework for self-supervised contrastive learning for continuous dependent data
AI总结：
本文提出面向连续依赖数据的对比式自监督学习理论框架，通过硬、软接近关系构造依赖感知损失，并以 Dependent TS2Vec 在多个时间及时空任务上取得显著性能提升。
摘要：
自监督学习（SSL）已成为一种强大的表示学习方法，尤其在计算机视觉领域。然而，其在时间和时空等依赖数据上的应用仍未得到充分研究。此外，传统对比式 SSL 方法通常假设样本之间存在语义独立性，而这对于表现出复杂相关性的依赖数据并不成立。我们提出了一种面向连续依赖数据的新型对比式 SSL 理论框架，允许最近的样本在语义上彼此接近。具体而言，我们提出了对象之间两种可能的真实相似度度量：硬接近和软接近。在此框架下，我们推导了估计相似度矩阵的解析形式，该矩阵同时容纳样本间的两类接近关系，从而引入具备依赖感知能力的损失函数。我们在时间和时空下游问题上验证了所提方法 Dependent TS2Vec。鉴于数据中呈现的依赖模式，我们的方法优于面向依赖数据的现代方法，突显了这些具有理论基础的 SSL 损失函数在捕获时空依赖方面的有效性。具体而言，我们在标准 UEA 和 UCR 基准上优于 TS2Vec，准确率分别提升 4.17% 和 2.08%。此外，在涉及复杂时空模式的干旱分类任务上，我们的方法取得了高 7% 的 ROC-AUC 分数。
英文摘要：
Self-supervised learning (SSL) has emerged as a powerful approach to learning representations, particularly in the field of computer vision. However, its application to dependent data, such as temporal and spatio-temporal domains, remains underexplored. Besides, traditional contrastive SSL methods often assume \emph{semantic independence between samples}, which does not hold for dependent data exhibiting complex correlations. We propose a novel theoretical framework for contrastive SSL tailored to \emph{continuous dependent data}, which allows the nearest samples to be semantically close to each other. In particular, we propose two possible \textit{ground truth similarity measures} between objects -- \emph{hard} and \emph{soft} closeness. Under it, we derive an analytical form for the \textit{estimated similarity matrix} that accommodates both types of closeness between samples, thereby introducing dependency-aware loss functions. We validate our approach, \emph{Dependent TS2Vec}, on temporal and spatio-temporal downstream problems. Given the dependency patterns presented in the data, our approach surpasses modern ones for dependent data, highlighting the effectiveness of our theoretically grounded loss functions for SSL in capturing spatio-temporal dependencies. Specifically, we outperform TS2Vec on the standard UEA and UCR benchmarks, with accuracy improvements of $4.17$\% and $2.08$\%, respectively. Furthermore, on the drought classification task, which involves complex spatio-temporal patterns, our method achieves a $7$\% higher ROC-AUC score.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
