TOTEM:面向通用时间序列分析的令牌化时间序列嵌入
TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis
AI总结:
本文提出TOTEM,通过自监督离散令牌化时间序列数据,构建跨多领域、多任务的通才时间序列模型,并在插补、异常检测和预测任务上以极少微调达到或超越现有最先进模型。
摘要:
本工作研究使用通才(或基础)模型进行时间序列分析的问题,这类模型是在许多数据领域上训练的模型。受大语言模型广泛成功的启发,我们考虑一种简单策略:通过自监督方式对来自大量数据集的时序数据进行离散令牌化,然后使用固定的令牌化方案在许多数据领域上解决各种任务。通常,时间序列模型要么在单个数据集上训练,要么以任务特定的方式构建(例如仅用于预测的模型),其中许多模型使用时间片段作为模型输入。因此,探索跨多个任务的高性能通才离散表示时间序列模型具有重要价值。我们的方法,令牌化时间序列嵌入(TOTEM),能够以极少或无需微调的方式生成此类通才时间序列模型,同时表现出强大的零样本性能。我们在三个常用且基于真实数据的时间序列任务上对TOTEM进行了近500次实验的广泛评估:插补(17个基线,12个数据集)、异常检测(19个基线,25个数据集)和预测(14个基线,12个数据集)。我们得出结论,TOTEM在典型的专家设置(即在一个领域上训练一个模型)以及通才设置(即在许多领域上训练单个模型)中均达到或超过现有最先进模型,这证明了令牌化在通用时间序列分析中的有效性。开源实现可在以下地址获取:https://github.com/SaberaTalukder/TOTEM;视频摘要可在以下地址获取:https://www.youtube.com/watch?v=OqrCpdb6MJk。
英文摘要:
This work studies the problem of time series analysis with generalist (or foundation) models, which are models trained across many data domains. Drawing inspiration from the widespread success of large language models, we consider the simple strategy of discretely tokenizing time series data drawn from a myriad of datasets via self-supervision, then using the fixed tokenization to solve a variety of tasks across many data domains. Canonically, time series models are either trained on a single dataset or built in a task-specific manner (e.g., a forecasting-only model), where many use patches of time as inputs to the model. As such, performant generalist, discrete representation time series models explored across many tasks are of value. Our method, TOkenized Time Series EMbeddings (TOTEM), produces such generalist time series models with minimal or no fine-tuning while exhibiting strong zero-shot performance. We evaluate TOTEM extensively over nearly 500 experiments on three commonly-studied time series tasks with real-world data: imputation (17 baselines, 12 datasets), anomaly detection (19 baselines, 25 datasets), and forecasting (14 baselines, 12 datasets). We conclude that TOTEM matches or outperforms existing state-of-the-art models in both the canonical specialist setting (i.e., training one model on one domain) as well as the generalist setting (i.e., training a single model on many domains), which demonstrates the efficacy of tokenization for general time series analysis. The open-source implementation is available here: https://github.com/SaberaTalukder/TOTEM; a video summary is available here: https://www.youtube.com/watch?v=OqrCpdb6MJk.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。