发表机构
RWTH Aachen University; Fraunhofer FIT（亚琛工业大学; 弗劳恩霍夫应用信息技术研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
SETTer通过解耦自注意力和混合掩码技术，在单层Transformer架构下有效捕获长期多元时间序列的时空模式，并在88%的基准场景中超越现有最优模型。
AI 中文摘要
长期多元时间序列在电力系统、交易等许多应用领域中扮演着重要角色。然而，由于它们通常表现出高维性和复杂的关系，传统预测方法对其准确预测相当困难。近期研究表明，基于Transformer的方法凭借其注意力机制在长期预测中相当有效。然而，在面对复杂的高维输入时，这些方法表现出过平滑、容量有限和不透明等问题。为此，本文引入了SETTer，一种基于Transformer的模型，通过引入解耦自注意力和混合掩码的新技术来解决这些挑战。所提出的技术使SETTer能够有效地捕获跨时间和通道维度的主要短期和长期模式。此外，我们通过简单的可解释结构丰富了模型层，这些结构指示了SETTer的判别模式。我们表明，采用单层Transformer架构，SETTer能够在不同数据复杂度下有效建模长期依赖关系。在用于长期多元时间序列预测的真实世界基准数据集上进行的大量实验表明，SETTer在88%的场景中优于最先进的模型。
英文摘要
Long-term multivariate time series plays a significant role in many application areas such as power systems, trading, etc. However, their accurate prediction is quite difficult for conventional forecasting methods as they often exhibit high dimensionality and complex relationships. Recent works show that transformer-based approaches are quite effective for long-term forecasting thanks to their attention mechanism. However, in the presence of complex high-dimensional inputs, they show evidence of oversmoothing, limited capacity, and opacity. To this end, this paper introduces SETTer, a transformer-based model that addresses these challenges by incorporating novel techniques for decoupled self-attention and hybrid masking. The proposed techniques enable SETTer to effectively capture the dominant short- and long-term patterns across the temporal and channel dimensions. In addition, we enrich the model layers with simple explainable structures that indicate the discriminative pattern of SETTer. We show that with a single-layer transformer architecture, SETTer can effectively model long-term dependencies in the presence of varying data complexities. Extensive experiments on real-word benchmark datasets for long-term multivariate time series forecasting demonstrate that SETTer outperforms state-of-the-art models in 88% of the scenarios.
