发表机构
Bar-Ilan University（巴伊兰大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
WinoTS提出基于小波的时频增强自蒸馏预训练，用于时间序列，在长期预测、零样本迁移和异常检测上超越基线，且线性探测可媲美监督模型。
AI 中文摘要
目前，时间序列模型的自监督预训练主要由下一标记预测和重建目标主导。在连续值域中，这些范式往往将模型容量浪费在高频、逐点噪声上，而牺牲了对不变结构的学习。虽然基于不变性的自蒸馏在计算机视觉中已被证明非常有效，但其在时间数据上的应用仍 largely 未被充分探索。有效地将此类方法适应于时间序列需要精心设计的增强：像裁剪这样的空间操作可能会改变重复周期的时间或扭曲信号，而基本的抖动可能提供的变异性有限。我们提出了用于时间序列的基于小波的自蒸馏（WinoTS），这是一种专门为时间信号设计的基于不变性的预训练范式。其核心在于，WinoTS利用时频增强来构建多尺度结构视图，而不会扭曲底层信号动态。在广泛的评估中，WinoTS在长期预测、跨域零样本迁移和无监督异常检测方面优于最先进的基线。值得注意的是，对冻结的WinoTS表示进行线性探测常常超过从头训练的完全监督模型。系统性消融研究表明，WinoTS是一个灵活的、与架构无关的框架，能在各种时间序列骨干网络上带来收益，并确立了时频变换作为视觉风格空间增强的原则性替代方案。
英文摘要
Self-supervised pre-training of time series models is currently dominated by next-token prediction and reconstruction objectives. In continuous-valued domains, these paradigms often waste model capacity on high-frequency, point-wise noise at the expense of learning invariant structure. While invariance-based self-distillation has proven highly effective in computer vision, its application to temporal data remains largely underexplored. Effectively adapting such methods to time series requires carefully designed augmentations: spatial operations like cropping can shift the timing of repeating cycles or distort the signal, while basic jittering may provide limited variation. We introduce Wavelet-based self-distillation for time series (WinoTS), an invariance-based pre-training paradigm designed specifically for temporal signals. At its core, WinoTS leverages time-frequency augmentations to construct multi-scale structural views without distorting underlying signal dynamics. Across extensive evaluations, WinoTS outperforms state-of-the-art baselines in long-term forecasting, cross-domain zero-shot transfer, and unsupervised anomaly detection. Notably, linear probing on frozen WinoTS representations frequently surpasses fully supervised models trained from scratch. Systematic ablations demonstrate that WinoTS is a flexible, architecture-agnostic framework yielding gains across time series backbones, and establish that time-frequency transformations provide a principled alternative to vision-style spatial augmentations.
