面向高维多元时间序列预测的可扩展Transformer
Scalable Transformer for High Dimensional Multivariate Time Series Forecasting
AI总结：
本文针对高维多元时间序列预测中通道依赖模型性能下降的问题，提出可扩展Transformer模型STHD，通过关系矩阵稀疏化、ReIndex训练策略和二维Transformer捕捉通道依赖，在三个高维数据集上取得显著改进。
摘要：
用于多元时间序列（MTS）预测的深度模型近年来取得了显著成功。通道依赖模型能够捕捉通道独立模型无法捕捉的复杂依赖关系。然而，现实应用中的通道数量超出了现有通道依赖模型的能力范围，并且与普遍预期相反，一些模型在处理高维数据时表现不如通道独立模型，这引发了人们对通道依赖模型性能的质疑。为了解决这一问题，本研究首先探究了这些通道依赖模型在高维MTS数据上表现欠佳的原因。我们的分析表明，两个主要问题在于：来自不相关序列的噪声引入增加了捕捉关键通道间依赖关系的难度，以及高维数据给训练策略带来的挑战。为解决这些问题，我们提出了STHD，即面向高维多元时间序列预测的可扩展Transformer。STHD包含三个组成部分：a) 关系矩阵稀疏化，用于限制引入的噪声并缓解内存问题；b) ReIndex作为一种训练策略，以实现更灵活的批量大小设置并增加训练数据的多样性；c) Transformer，用于处理二维输入并捕捉通道依赖关系。这些组件共同使STHD能够在保持计算可行性的同时处理高维MTS。此外，实验结果表明，STHD在三个高维数据集Crime-Chicago、Wiki-People和Traffic上均有显著改进。源代码和数据集已公开，地址为https://github.com/xinzzzhou/ScalableTransformer4HighDimensionMTSF.git。
英文摘要：
Deep models for Multivariate Time Series (MTS) forecasting have recently demonstrated significant success. Channel-dependent models capture complex dependencies that channel-independent models cannot capture. However, the number of channels in real-world applications outpaces the capabilities of existing channel-dependent models, and contrary to common expectations, some models underperform the channel-independent models in handling high-dimensional data, which raises questions about the performance of channel-dependent models. To address this, our study first investigates the reasons behind the suboptimal performance of these channel-dependent models on high-dimensional MTS data. Our analysis reveals that two primary issues lie in the introduced noise from unrelated series that increases the difficulty of capturing the crucial inter-channel dependencies, and challenges in training strategies due to high-dimensional data. To address these issues, we propose STHD, the Scalable Transformer for High-Dimensional Multivariate Time Series Forecasting. STHD has three components: a) Relation Matrix Sparsity that limits the noise introduced and alleviates the memory issue; b) ReIndex applied as a training strategy to enable a more flexible batch size setting and increase the diversity of training data; and c) Transformer that handles 2-D inputs and captures channel dependencies. These components jointly enable STHD to manage the high-dimensional MTS while maintaining computational feasibility. Furthermore, experimental results show STHD's considerable improvement on three high-dimensional datasets: Crime-Chicago, Wiki-People, and Traffic. The source code and dataset are publicly available https://github.com/xinzzzhou/ScalableTransformer4HighDimensionMTSF.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
