arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.20150cs.LGcs.AIcs.CY
收藏

TFB:迈向全面且公平的时间序列预测方法基准测试

TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods

East China Normal University, China(东中国师范大学);Huawei Cloud Algorithm Innovation Lab, China(华为云算法创新实验室);Aalborg University, Denmark(奥尔堡大学)

Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, Bin Yang

AI总结:

本文提出TFB自动化基准,通过覆盖10个领域数据集、纳入统计/机器学习/深度学习等多样方法及灵活可扩展流程,消除传统方法偏见,并对21种单变量和14种多变量时间序列预测方法进行了大规模公平评估。

摘要:

时间序列在经济、交通、健康和能源等多个领域中产生,其中对未来值的预测具有众多重要应用。毫不奇怪,许多预测方法正被不断提出。为确保进展,能够以全面且可靠的方式对这些方法进行实证研究和比较至关重要。为实现这一目标,我们提出了TFB,一个面向时间序列预测(TSF)方法的自动化基准测试。TFB通过解决与数据集、比较方法和评估流程相关的不足,推动了该领域的最新进展:1)数据领域覆盖不足,2)对传统方法的刻板偏见,以及3)流程不一致且缺乏灵活性。为了实现更好的领域覆盖,我们纳入了来自10个不同领域的数据集:交通、电力、能源、环境、自然、经济、股票市场、银行、健康和网络。我们还提供了时间序列特征描述,以确保所选数据集的全面性。为了消除对某些方法的偏见,我们纳入了多样化的方法,包括统计学习、机器学习和深度学习方法,并且我们还支持多种评估策略和指标,以确保对不同方法进行更全面的评估。为了支持将不同方法集成到基准测试中并实现公平比较,TFB具有灵活且可扩展的流程,可消除偏见。接下来,我们利用TFB对8,068条单变量时间序列上的21种单变量时间序列预测(UTSF)方法以及25个数据集上的14种多变量时间序列预测(MTSF)方法进行了全面评估。基准测试代码和数据可在 https://github.com/decisionintelligence/TFB 获取。我们还推出了在线时间序列排行榜:https://decisionintelligence.github.io/OpenTS/OpenTS-Bench/。

英文摘要:

Time series are generated in diverse domains such as economic, traffic, health, and energy, where forecasting of future values has numerous important applications. Not surprisingly, many forecasting methods are being proposed. To ensure progress, it is essential to be able to study and compare such methods empirically in a comprehensive and reliable manner. To achieve this, we propose TFB, an automated benchmark for Time Series Forecasting (TSF) methods. TFB advances the state-of-the-art by addressing shortcomings related to datasets, comparison methods, and evaluation pipelines: 1) insufficient coverage of data domains, 2) stereotype bias against traditional methods, and 3) inconsistent and inflexible pipelines. To achieve better domain coverage, we include datasets from 10 different domains: traffic, electricity, energy, the environment, nature, economic, stock markets, banking, health, and the web. We also provide a time series characterization to ensure that the selected datasets are comprehensive. To remove biases against some methods, we include a diverse range of methods, including statistical learning, machine learning, and deep learning methods, and we also support a variety of evaluation strategies and metrics to ensure a more comprehensive evaluations of different methods. To support the integration of different methods into the benchmark and enable fair comparisons, TFB features a flexible and scalable pipeline that eliminates biases. Next, we employ TFB to perform a thorough evaluation of 21 Univariate Time Series Forecasting (UTSF) methods on 8,068 univariate time series and 14 Multivariate Time Series Forecasting (MTSF) methods on 25 datasets. The benchmark code and data are available at https://github.com/decisionintelligence/TFB. We have also launched an online time series leaderboard: https://decisionintelligence.github.io/OpenTS/OpenTS-Bench/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑