arXiv 2405.151242024-11-09cs.LGcs.AI

时间序列预测的缩放定律

Scaling Law for Time Series Forecasting

Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Zhili College, Tsinghua University（清华大学致理书院）；University of Copenhagen（哥本哈根大学）；University of Washington（华盛顿大学）

Jingzhe Shi, Qinwei Ma, Huan Ma, Lei Li

备注：: Accepted by NeurIPS 2024

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出时间序列预测的缩放定律理论，综合考虑数据集大小、模型复杂度及未被探索的回溯范围，解释了模型能力与输入范围对性能的异常影响，并通过多数据集实证验证了该理论框架。

摘要：

奖励大型数据集、复杂模型和增强数据粒度的缩放定律已在深度学习的各个领域中被观察到。然而，针对时间序列预测的研究对深度学习方法在时间序列预测中的缩放行为提出了质疑：虽然更多的训练数据能提升性能，但能力更强的模型并不总是优于能力较弱的模型，且更长的输入范围可能会损害某些模型的性能。我们提出了一种时间序列预测的缩放定律理论，能够解释这些看似异常的行为。我们考虑了数据集大小和模型复杂度的影响，以及时间序列数据粒度，特别关注了回溯范围，这是以往理论中未被探索的方面。此外，我们使用多样化的时间序列预测数据集对各种模型进行了实证评估，这(1)验证了在时间序列预测领域内关于数据集大小和模型复杂度的缩放定律的有效性，以及(2)验证了我们的理论框架，特别是关于回溯范围的影响。我们希望我们的发现能启发针对有限规模时间序列预测数据集的新模型，以及未来工作中用于时间序列预测的大型基础数据集和模型。实验代码已公开于https://github.com/JingzheShi/ScalingLawForTimeSeriesForecasting。

英文摘要：

Scaling law that rewards large datasets, complex models and enhanced data granularity has been observed in various fields of deep learning. Yet, studies on time series forecasting have cast doubt on scaling behaviors of deep learning methods for time series forecasting: while more training data improves performance, more capable models do not always outperform less capable models, and longer input horizons may hurt performance for some models. We propose a theory for scaling law for time series forecasting that can explain these seemingly abnormal behaviors. We take into account the impact of dataset size and model complexity, as well as time series data granularity, particularly focusing on the look-back horizon, an aspect that has been unexplored in previous theories. Furthermore, we empirically evaluate various models using a diverse set of time series forecasting datasets, which (1) verifies the validity of scaling law on dataset size and model complexity within the realm of time series forecasting, and (2) validates our theoretical framework, particularly regarding the influence of look back horizon. We hope our findings may inspire new models targeting time series forecasting datasets of limited size, as well as large foundational datasets and models for time series forecasting in future work. Code for our experiments has been made public at https://github.com/JingzheShi/ScalingLawForTimeSeriesForecasting.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。