arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2009.14799cs.LGstat.ML
收藏

MQTransformer:具有上下文相关和反馈感知注意力的多水平预测

MQTransformer: Multi-Horizon Forecasts with Context Dependent and Feedback-Aware Attention

Amazon(亚马逊公司)

Carson Eisenach, Yagna Patel, Dhruv Madeka

摘要:

神经预测的最新进展已经在概率需求预测的准确性方面产生了重大改进。在这项工作中,我们提出了新的改进,目前的最先进的结合变化的启发,在Transformer架构的自然语言处理的最新进展。我们开发了一种新的解码器-编码器注意上下文对齐,通过允许网络根据它正在产生预测的上下文研究自己的历史来提高预测准确性。我们还提出了一种新的位置编码,允许神经网络学习上下文相关的季节性函数以及任意的假期距离。最后,我们表明,目前最先进的MQ-预测器(温等人,2017年)模型显示过度的可变性,未能利用预测中以前的错误来提高准确性。我们提出了一种新的解码器-自注意力预测计划,产生显着改善的预测过度变化。

英文摘要:

Recent advances in neural forecasting have produced major improvements in accuracy for probabilistic demand prediction. In this work, we propose novel improvements to the current state of the art by incorporating changes inspired by recent advances in Transformer architectures for Natural Language Processing. We develop a novel decoder-encoder attention for context-alignment, improving forecasting accuracy by allowing the network to study its own history based on the context for which it is producing a forecast. We also present a novel positional encoding that allows the neural network to learn context-dependent seasonality functions as well as arbitrary holiday distances. Finally we show that the current state of the art MQ-Forecaster (Wen et al., 2017) models display excess variability by failing to leverage previous errors in the forecast to improve accuracy. We propose a novel decoder-self attention scheme for forecasting that produces significant improvements in the excess variation of the forecast.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑