走非规则路线：增强时序预测 Transformer 的解码器
Take an Irregular Route: Enhance the Decoder of Time-Series Forecasting Transformer
AI总结：
针对时序预测 Transformer 忽视解码器的问题，本文提出 FPPformer，通过自底向上编码器、自顶向下解码器及两类注意力结合增强建模，并在十二个基准上验证其优越性能。
摘要：
随着物联网（IoT）系统的发展，精确的长期预测方法对于决策者评估当前状态并制定未来策略必不可少。目前，Transformer 和 MLP 是深度时间序列预测的两种范式，前者凭借精巧的注意力机制和编码器-解码器架构而更为流行。然而，数据科学家似乎更愿意深入研究编码器，对解码器则漠不关心。一些研究者甚至采用线性投影代替解码器以降低复杂度。我们认为，提取输入序列特征以及寻找输入序列与预测序列之间的关系——分别对应编码器和解码器的功能——都至关重要。受 FPN 在 CV 领域成功的启发，我们提出 FPPformer，在编码器和解码器中分别采用自底向上和自顶向下的架构，以构建完整且合理的层次结构。本文利用前沿的 patch-wise attention，并将其与改进后的 element-wise attention 相结合而作进一步发展；这种结合在编码器和解码器中的形式也有所不同。在十二个基准上与六个最先进基线进行的大量实验，验证了 FPPformer 的良好性能，以及在时间序列预测 Transformer 中精心设计解码器的重要性。源代码已发布于 https://github.com/OrigamiSL/FPPformer。
英文摘要：
With the development of Internet of Things (IoT) systems, precise long-term forecasting method is requisite for decision makers to evaluate current statuses and formulate future policies. Currently, Transformer and MLP are two paradigms for deep time-series forecasting and the former one is more prevailing in virtue of its exquisite attention mechanism and encoder-decoder architecture. However, data scientists seem to be more willing to dive into the research of encoder, leaving decoder unconcerned. Some researchers even adopt linear projections in lieu of the decoder to reduce the complexity. We argue that both extracting the features of input sequence and seeking the relations of input and prediction sequence, which are respective functions of encoder and decoder, are of paramount significance. Motivated from the success of FPN in CV field, we propose FPPformer to utilize bottom-up and top-down architectures respectively in encoder and decoder to build the full and rational hierarchy. The cutting-edge patch-wise attention is exploited and further developed with the combination, whose format is also different in encoder and decoder, of revamped element-wise attention in this work. Extensive experiments with six state-of-the-art baselines on twelve benchmarks verify the promising performances of FPPformer and the importance of elaborately devising decoder in time-series forecasting Transformer. The source code is released in https://github.com/OrigamiSL/FPPformer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
