arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.05713cs.LG
收藏

tsGT:基于 Transformer 的随机时间序列建模

tsGT: Stochastic Time Series Modeling With Transformer

IDEAS NCBR;Institute of Mathematics, Polish Academy of Sciences(波兰科学院数学研究所);University of Warsaw(华沙大学);Jagiellonian University(雅盖隆大学);OpenAI

Łukasz Kuciński, Witold Drzewakowski, Mateusz Olko, Piotr Kozakowski, Łukasz Maziarka, Marta Emilia Nowakowska, Łukasz Kaiser, Piotr Miłoś

AI总结:

本文提出基于通用 Transformer 的随机时间序列模型 tsGT,采用滚动窗口回测协议,在四个常用数据集上于 MAD、RMSE、QL 和 CRPS 指标中取得优于现有模型的表现。

摘要:

时间序列方法在几乎所有处理时间结构化数据的科学领域都具有根本重要性。近来,带有时间序列专用架构偏置的确定性 transformer 模型大量涌现。本文中,我们走向不同方向,提出 tsGT,这是一种构建于通用 transformer 架构之上的随机时间序列模型。我们专注于使用广为人知且具有理论依据的滚动窗口回测与评估协议。结果表明,在四个常用数据集上,tsGT 在 MAD 和 RMSE 上优于最先进模型,并在 QL 和 CRPS 上超过同类随机模型。我们还通过详细分析 tsGT 建模数据分布及预测边际分位数值的能力,对这些结果进行了补充。

英文摘要:

Time series methods are of fundamental importance in virtually any field of science that deals with temporally structured data. Recently, there has been a surge of deterministic transformer models with time series-specific architectural biases. In this paper, we go in a different direction by introducing tsGT, a stochastic time series model built on a general-purpose transformer architecture. We focus on using a well-known and theoretically justified rolling window backtesting and evaluation protocol. We show that tsGT outperforms the state-of-the-art models on MAD and RMSE, and surpasses its stochastic peers on QL and CRPS, on four commonly used datasets. We complement these results with a detailed analysis of tsGT's ability to model the data distribution and predict marginal quantile values.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑