arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.02068cs.LGcs.AI
收藏

长期时间序列预测:特征、模式和分布

Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution

Baidu Research(百度研究院);Zhejiang University(浙江大学);Tsinghua University(清华大学);China Longyuan Power Group Corp. Ltd.(中国龙源电力集团股份有限公司);Dalian University of Technology(大连理工大学)

Yan Li, Xinjiang Lu, Haoyi Xiong, Jian Tang, Jiantao Su, Bo Jin, Dejing Dou

摘要:

长期时间序列预测(LTTF)已成为许多应用中的迫切需求,如风力发电规划。Transformer模型由于具有高度的计算自注意机制而被采用以提供高预测能力。虽然人们可以通过引入LTTF的点式自注意的稀疏性来降低Transformers的复杂性,但是有限的信息利用阻止了模型全面地探索复杂的依赖关系。为此,我们提出了一种基于变换器的高效LTTF模型Conformer,该模型与现有的LTTF方法相比有三个不同之处:(1)在滑动窗口注意力和稳态瞬时递归网络(SIRN)的基础上,提出了一种在不牺牲信息利用率的前提下具有线性复杂度的编解码器结构;(ii)设计了一个基于归一化流程的模块,通过直接用SIRN中的潜变量推断输出,进一步提高了信息利用率;(iii)时间序列数据中的序列间相关性和时间动态被明确地建模以促进下游自注意机制。在七个真实数据集上的大量实验表明,Conformer在LTTF上的性能优于最先进的方法,并通过不确定性量化生成可靠的预测结果。

英文摘要:

Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. Though one could lower the complexity of Transformers by inducing the sparsity in point-wise self-attentions for LTTF, the limited information utilization prohibits the model from exploring the complex dependencies comprehensively. To this end, we propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects: (i) an encoder-decoder architecture incorporating a linear complexity without sacrificing information utilization is proposed on top of sliding-window attention and Stationary and Instant Recurrent Network (SIRN); (ii) a module derived from the normalizing flow is devised to further improve the information utilization by inferring the outputs with the latent variables in SIRN directly; (iii) the inter-series correlation and temporal dynamics in time-series data are modeled explicitly to fuel the downstream self-attention mechanism. Extensive experiments on seven real-world datasets demonstrate that Conformer outperforms the state-of-the-art methods on LTTF and generates reliable prediction results with uncertainty quantification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑