arXiv 2310.050632023-12-05cs.LG

在 CloudOps 领域推动时间序列预测预训练的极限

Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domain

Salesforce AI Research（赛富时人工智能研究院）；Singapore Management University（新加坡管理大学）

Gerald Woo, Chenghao Liu, Akshat Kumar, Doyen Sahoo

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文针对时间序列预训练研究受限于小规模数据集的问题，引入CloudOps领域三个大规模预测数据集（最大含数十亿观测值），并验证了候选架构的零样本能力与扩展收益，在最大数据集上实现27%误差降低。

摘要：

时间序列在预训练和迁移学习时代已经落后。虽然自然语言处理和计算机视觉领域的研究正在享受越来越大的数据集来训练大规模模型，但最流行的时间序列数据集仅包含数万个时间步，限制了我们研究预训练和扩展有效性的能力。最近的研究也对表达性模型和规模的必要性提出了质疑。为了缓解这些问题，我们引入了来自云运营（CloudOps）领域的三个大规模时间序列预测数据集，其中最大的数据集拥有数十亿个观测值，从而能够进一步研究时间序列模型的预训练和扩展。我们通过识别一个有前景的候选架构，为研究时间序列模型的预训练和扩展建立了实证基础，并为未来研究铺平了道路。我们表明，它是一个强大的零样本基线，并且受益于模型和数据集规模的进一步扩展。伴随这些数据集和结果的是一套全面的基准测试结果，将经典和深度学习基线与我们预训练的方法进行比较——在最大的数据集上实现了27%的误差降低。代码和数据集可在 https://github.com/SalesforceAIResearch/pretrain-time-series-cloudops 找到。

英文摘要：

Time series has been left behind in the era of pre-training and transfer learning. While research in the fields of natural language processing and computer vision are enjoying progressively larger datasets to train massive models, the most popular time series datasets consist of only tens of thousands of time steps, limiting our ability to study the effectiveness of pre-training and scaling. Recent studies have also cast doubt on the need for expressive models and scale. To alleviate these issues, we introduce three large-scale time series forecasting datasets from the cloud operations (CloudOps) domain, the largest having billions of observations, enabling further study into pre-training and scaling of time series models. We build the empirical groundwork for studying pre-training and scaling of time series models and pave the way for future research by identifying a promising candidate architecture. We show that it is a strong zero-shot baseline and benefits from further scaling, both in model and dataset size. Accompanying these datasets and results is a suite of comprehensive benchmark results comparing classical and deep learning baselines to our pre-trained method - achieving a 27% reduction in error on the largest dataset. Code and datasets can be found https://github.com/SalesforceAIResearch/pretrain-time-series-cloudops.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。