通过同时进行低频和高频特征自举的大规模时间序列表示学习
Large Scale Time-Series Representation Learning via Simultaneous Low and High Frequency Feature Bootstrapping
AI总结：
针对无标签时间序列表示学习难以同时捕获高低频特征且计算成本高的问题，提出一种非对比自监督方法，通过结合MLP和TCN头互补学习低高频时变特征，在五个真实数据集上达到最先进性能。
摘要：
从无标签时间序列数据中学习表示是一个具有挑战性的问题。时间序列领域的大多数现有自监督和无监督方法无法同时捕获低频和高频特征。此外，其中一些方法采用了如transformers的大规模模型，或依赖于计算成本高昂的技术，如对比学习。为了解决这些问题，我们提出了一种非对比自监督学习方法，以经济高效的方式有效捕获低频和高频时变特征。我们的方法以原始时间序列数据作为输入，通过从同一增强族中随机采样增强，为模型的两个分支创建两个不同的增强视图。沿用BYOL的术语，这两个分支被称为online和target网络，允许对潜在表示进行自举。与BYOL中主干编码器后接多层感知机（MLP）头不同，所提出的模型包含额外的时序卷积网络（TCN）头。由于增强视图通过编码器的大卷积核卷积块，MLP和TCN的后续组合因感受野的不同，能够实现对低频和高频时变特征的有效表示。这两个模块（MLP和TCN）以互补的方式起作用。我们训练online网络，其中每个模块学习预测target网络分支中相应模块的输出。为了证明我们模型的鲁棒性，我们在五个真实世界时间序列数据集上进行了广泛的实验和消融研究。我们的方法在所有五个真实世界数据集上都达到了最先进的性能。
英文摘要：
Learning representation from unlabeled time series data is a challenging problem. Most existing self-supervised and unsupervised approaches in the time-series domain do not capture low and high-frequency features at the same time. Further, some of these methods employ large scale models like transformers or rely on computationally expensive techniques such as contrastive learning. To tackle these problems, we propose a non-contrastive self-supervised learning approach efficiently captures low and high-frequency time-varying features in a cost-effective manner. Our method takes raw time series data as input and creates two different augmented views for two branches of the model, by randomly sampling the augmentations from same family. Following the terminology of BYOL, the two branches are called online and target network which allows bootstrapping of the latent representation. In contrast to BYOL, where a backbone encoder is followed by multilayer perceptron (MLP) heads, the proposed model contains additional temporal convolutional network (TCN) heads. As the augmented views are passed through large kernel convolution blocks of the encoder, the subsequent combination of MLP and TCN enables an effective representation of low as well as high-frequency time-varying features due to the varying receptive fields. The two modules (MLP and TCN) act in a complementary manner. We train an online network where each module learns to predict the outcome of the respective module of target network branch. To demonstrate the robustness of our model we performed extensive experiments and ablation studies on five real-world time-series datasets. Our method achieved state-of-art performance on all five real-world datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
