arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.11470cs.LGcs.AI

VCFormer:用于多元时间序列预测的具有固有滞后相关性的变量相关性 Transformer

VCformer: Variable Correlation Transformer with Inherent Lagged Correlation for Multivariate Time Series Forecasting

College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机与软件学院);National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University(深圳大学国家大数据系统计算技术工程实验室)

Yingnan Yang, Qingling Zhu, Jianyong Chen

AI总结:

本文提出 VCFormer,通过可整合滞后互相关的 VCA 模块与基于 Koopman 理论的 KTD 模块,同时建模变量关系和时间依赖,并在八个真实数据集上达到先进预测性能。

摘要:

多元时间序列(MTS)预测已被广泛应用于天气预报、能源消耗等多个领域。然而,当前研究仍依赖普通的逐点自注意力机制来捕捉跨变量依赖,这不足以提取变量之间隐含的复杂互相关关系。为弥补这一空白,我们提出了 Variable Correlation Transformer(VCFormer),它利用 Variable Correlation Attention(VCA)模块挖掘变量之间的相关性。具体而言,VCA 基于随机过程理论,计算并整合 queries 与 keys 之间不同滞后对应的互相关分数,从而增强其发现多元关系的能力。此外,受 Koopman 动力学理论启发,我们还开发了 Koopman Temporal Detector(KTD),以更好地处理时间序列中的非平稳性。这两个关键组件使 VCFormer 能够同时提取多元相关性和时间依赖关系。我们在八个真实世界数据集上进行的大量实验证明了 VCFormer 的有效性;与其他最先进的基线模型相比,其取得了顶尖性能。代码可在该仓库获取:https://github.com/CSyyn/VCformer。

英文摘要:

Multivariate time series (MTS) forecasting has been extensively applied across diverse domains, such as weather prediction and energy consumption. However, current studies still rely on the vanilla point-wise self-attention mechanism to capture cross-variable dependencies, which is inadequate in extracting the intricate cross-correlation implied between variables. To fill this gap, we propose Variable Correlation Transformer (VCformer), which utilizes Variable Correlation Attention (VCA) module to mine the correlations among variables. Specifically, based on the stochastic process theory, VCA calculates and integrates the cross-correlation scores corresponding to different lags between queries and keys, thereby enhancing its ability to uncover multivariate relationships. Additionally, inspired by Koopman dynamics theory, we also develop Koopman Temporal Detector (KTD) to better address the non-stationarity in time series. The two key components enable VCformer to extract both multivariate correlations and temporal dependencies. Our extensive experiments on eight real-world datasets demonstrate the effectiveness of VCformer, achieving top-tier performance compared to other state-of-the-art baseline models. Code is available at this repository: https://github.com/CSyyn/VCformer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑