arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2110.08255cs.LG
收藏

Yformer:基于U-Net的Transformer架构,用于远时域时间序列预测

Yformer: U-Net Inspired Transformer Architecture for Far Horizon Time Series Forecasting

University of Hildesheim(希尔德斯海姆大学);Technische Universität Berlin(柏林工业大学)

Kiran Madhusudhanan, Johannes Burchert, Nghia Duong-Trung, Stefan Born, Lars Schmidt-Thieme

摘要:

时间序列数据在研究和各种工业应用中无处不在。有效地分析可用的历史数据并提供对遥远未来的见解,使我们能够做出有效的决策。最近的研究已经证明了基于变压器的架构的优越性能,特别是在远地平线时间序列预测的制度。然而,现有技术的稀疏Transformer架构的当前状态不能耦合下采样和上采样过程以产生与输入类似分辨率的输出。我们提出了Yformer模型,基于一种新的Y形编码器-解码器架构,该架构(1)在U-Net启发的架构中使用从缩小的编码器层到相应的上采样解码器层的直接连接,(2)将缩小/上采样与稀疏注意力相结合以捕获长距离效果,以及(3)通过添加辅助重建损失来稳定编码器-解码器堆栈。在四个基准数据集上使用相关基线进行了广泛的实验,证明与单变量和多变量设置的当前最新技术水平相比,平均改善了19.82,18.41%的MSE和13.62,11.85%的MAE。

英文摘要:

Time series data is ubiquitous in research as well as in a wide variety of industrial applications. Effectively analyzing the available historical data and providing insights into the far future allows us to make effective decisions. Recent research has witnessed the superior performance of transformer-based architectures, especially in the regime of far horizon time series forecasting. However, the current state of the art sparse Transformer architectures fail to couple down- and upsampling procedures to produce outputs in a similar resolution as the input. We propose the Yformer model, based on a novel Y-shaped encoder-decoder architecture that (1) uses direct connection from the downscaled encoder layer to the corresponding upsampled decoder layer in a U-Net inspired architecture, (2) Combines the downscaling/upsampling with sparse attention to capture long-range effects, and (3) stabilizes the encoder-decoder stacks with the addition of an auxiliary reconstruction loss. Extensive experiments have been conducted with relevant baselines on four benchmark datasets, demonstrating an average improvement of 19.82, 18.41 percentage MSE and 13.62, 11.85 percentage MAE in comparison to the current state of the art for the univariate and the multivariate settings respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑