arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.03885cs.LGcs.AI
收藏

MOMENT:一个开放时间序列基础模型家族

MOMENT: A Family of Open Time-series Foundation Models

University of Pennsylvania(宾夕法尼亚大学)

Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, Artur Dubrawski

AI总结:

本文提出开源时间序列基础模型家族MOMENT,通过构建大型公共时间序列库Time series Pile解决多数据集预训练挑战,并设计基准验证其在有限监督下多任务的有效性。

摘要:

我们介绍了MOMENT,一个用于通用时间序列分析的开源基础模型家族。由于(1)缺乏一个大型且统一的公共时间序列库,以及(2)多样的时间序列特性使得多数据集训练变得繁重,在时间序列数据上预训练大型模型具有挑战性。此外,(3)用于评估这些模型的实验基准,特别是在资源、时间和监督有限的情况下,仍处于起步阶段。为了应对这些挑战,我们编制了一个大型且多样化的公共时间序列集合,称为Time series Pile,并系统地解决时间序列特有的挑战,以实现大规模多数据集预训练。最后,我们基于最近的工作设计了一个基准,以在有限监督设置下评估时间序列基础模型在多样任务和数据集上的表现。在该基准上的实验证明了我们的预训练模型在最小数据和任务特定微调下的有效性。最后,我们提出了关于大型预训练时间序列模型的几个有趣的实证观察。预训练模型(AutonLab/MOMENT-1-large)和Time Series Pile(AutonLab/Timeseries-PILE)可在Huggingface上获取。

英文摘要:

We introduce MOMENT, a family of open-source foundation models for general-purpose time series analysis. Pre-training large models on time series data is challenging due to (1) the absence of a large and cohesive public time series repository, and (2) diverse time series characteristics which make multi-dataset training onerous. Additionally, (3) experimental benchmarks to evaluate these models, especially in scenarios with limited resources, time, and supervision, are still in their nascent stages. To address these challenges, we compile a large and diverse collection of public time series, called the Time series Pile, and systematically tackle time series-specific challenges to unlock large-scale multi-dataset pre-training. Finally, we build on recent work to design a benchmark to evaluate time series foundation models on diverse tasks and datasets in limited supervision settings. Experiments on this benchmark demonstrate the effectiveness of our pre-trained models with minimal data and task-specific fine-tuning. Finally, we present several interesting empirical observations about large pre-trained time series models. Pre-trained models (AutonLab/MOMENT-1-large) and Time Series Pile (AutonLab/Timeseries-PILE) are available on Huggingface.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑