Gateformer：通过时间与变量维度注意力及门控表示推进多元时间序列预测
Gateformer: Advancing Multivariate Time Series Forecasting through Temporal and Variate-Wise Attention with Gated Representations
AI总结：
Gateformer通过变量独立嵌入、跨时间与跨变量注意力及门控机制，在13个数据集上达到最优性能，并可集成到其他预测器中提升最高20.7%的效果。
摘要：
近年来，使用Transformer架构进行时间序列建模的研究兴趣激增。然而，用Transformer预测多元时间序列面临独特挑战，因为它需要同时建模时间（跨时间）依赖和变量（跨变量）依赖。尽管基于Transformer的模型因其在捕捉序列关系和跨变量关系方面的灵活性而广受欢迎，但在Transformer架构背景下如何最佳地整合这两类信息，同时兼顾性能和效率，仍不清楚。我们重新设计Transformer架构，以有效建模跨时间和跨变量依赖。我们的方法首先将每个变量独立嵌入到捕捉其跨时间动态的变量维度表示中，然后通过对这些学习到的嵌入进行注意力机制来建模跨变量依赖。在跨时间和跨变量建模阶段的门控操作调节信息流动，使模型能够聚焦于最相关的特征以实现准确预测。我们的方法在13个真实世界数据集上取得了最先进的性能，并且可以无缝集成到其他基于Transformer和基于LLM的预测器中，相比原始模型性能提升最高达20.7%。代码可在以下仓库获取：https://github.com/nyuolab/Gateformer。
英文摘要：
There has been a recent surge of interest in time series modeling using the Transformer architecture. However, forecasting multivariate time series with Transformer presents a unique challenge as it requires modeling both temporal (cross-time) and variate (cross-variate) dependencies. While Transformer-based models have gained popularity for their flexibility in capturing both sequential and cross-variate relationships, it is unclear how to best integrate these two sources of information in the context of the Transformer architecture while optimizing for both performance and efficiency. We re-purpose the Transformer architecture to effectively model both cross-time and cross-variate dependencies. Our approach begins by embedding each variate independently into a variate-wise representation that captures its cross-time dynamics, and then models cross-variate dependencies through attention mechanisms on these learned embeddings. Gating operations in both cross-time and cross-variate modeling phases regulate information flow, allowing the model to focus on the most relevant features for accurate predictions. Our method achieves state-of-the-art performance across 13 real-world datasets and can be seamlessly integrated into other Transformer-based and LLM-based forecasters, delivering performance improvements up to 20.7\% over original models. Code is available at this repository: https://github.com/nyuolab/Gateformer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
