arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.14045cs.LGcs.AI
收藏

LTSM-Bundle: 一个用于时间序列预测的大型语言模型工具包和基准

LTSM-Bundle: A Toolbox and Benchmark on Large Language Models for Time Series Forecasting

Yu-Neng Chuang, Songchen Li, Jiayi Yuan, Guanchu Wang, Kwei-Herng Lai, Joshua Han, Zihang Xu, Songyuan Sui, Leisheng Yu, Sirui Ding, Chia-Yuan Chang, Alfredo Co… 展开作者

Yu-Neng Chuang, Songchen Li, Jiayi Yuan, Guanchu Wang, Kwei-Herng Lai, Joshua Han, Zihang Xu, Songyuan Sui, Leisheng Yu, Sirui Ding, Chia-Yuan Chang, Alfredo Costilla Reyes, Daochen Zha, Xia Hu

AI总结:

LTSM-Bundle提出了一种用于时间序列预测的大型语言模型工具包和基准,通过模块化和多维度基准测试,结合最优设计选择,提升了零样本和少量样本性能。

摘要:

时间序列预测(TSF)长期以来一直是时间序列分析中的挑战。受大型语言模型(LLMs)成功启发,研究者现在正在开发大型时间序列模型(LTSMs)——通用的变压器模型,通过自回归预测来改进TSF。然而,将LTSMs训练在异构时间序列数据上提出了独特的挑战,包括数据集中的多样的频率、维度和模式。最近的研究已经研究并评估了各种设计选择,旨在增强LTSMs的训练和泛化能力。然而,这些设计选择通常是孤立地研究和评估的,并未进行整体基准测试。在本工作中,我们引入了LTSM-Bundle,一个全面的工具包和基准,涵盖预处理技术、模型配置和数据集配置。它从多个维度模块化并基准测试了LTSMs,包括提示策略、分词方法、训练范式、基础模型选择、数据量和数据集多样性。此外,我们结合了我们在研究中发现的最有效的设计选择。实证结果表明,这种组合在基准数据集上相比最先进的LTSMs和传统TSF方法在零样本和少量样本性能上表现更优。

英文摘要:

Time Series Forecasting (TSF) has long been a challenge in time series analysis. Inspired by the success of Large Language Models (LLMs), researchers are now developing Large Time Series Models (LTSMs)-universal transformer-based models that use autoregressive prediction-to improve TSF. However, training LTSMs on heterogeneous time series data poses unique challenges, including diverse frequencies, dimensions, and patterns across datasets. Recent endeavors have studied and evaluated various design choices aimed at enhancing LTSM training and generalization capabilities. However, these design choices are typically studied and evaluated in isolation and are not benchmarked collectively. In this work, we introduce LTSM-Bundle, a comprehensive toolbox, and benchmark for training LTSMs, spanning pre-processing techniques, model configurations, and dataset configuration. It modularized and benchmarked LTSMs from multiple dimensions, encompassing prompting strategies, tokenization approaches, training paradigms, base model selection, data quantity, and dataset diversity. Furthermore, we combine the most effective design choices identified in our study. Empirical results demonstrate that this combination achieves superior zero-shot and few-shot performances compared to state-of-the-art LTSMs and traditional TSF methods on benchmark datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑