arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.17703cs.LG
收藏

PGN:RNN的新继任者在长程时间序列预测中非常有效

PGN: The RNN's New Successor is Effective for Long-Range Time Series Forecasting

School of Computer and Information Technology, Beijing Jiaotong University(北京交通大学计算机与信息技术学院);School of Cyberspace Science and Technology, Beijing Jiaotong University(北京交通大学网络空间科学与技术学院)

Yuxin Jia, Youfang Lin, Jing Yu, Shuo Wang, Tianhao Liu, Huaiyu Wan

AI总结:

提出RNN新继任者PGN,通过HIE层和门控机制将信息传播路径降至O(1),并构建双分支框架TPGN全面捕获时序语义,在五个数据集上实现SOTA性能与高效率。

摘要:

由于RNN的循环结构,过长的信息传播路径在捕捉长程依赖性、梯度爆炸/消失问题以及低效的顺序执行方面带来了限制。基于此,我们提出了一种名为Parallel Gated Network (PGN)的新范式,作为RNN的新继任者。PGN通过设计的Historical Information Extraction (HIE)层直接从先前时间步捕获信息,并利用门控机制将其与当前时间步信息进行选择和融合。这将信息传播路径降至$\mathcal{O}(1)$,有效解决了RNN的局限性。为了增强PGN在长程时间序列预测任务中的性能,我们提出了一种名为Temporal PGN (TPGN)的新颖时间建模框架。TPGN结合两个分支来全面捕获时间序列的语义信息。一个分支利用PGN捕获长程周期模式,同时保留其局部特征。另一个分支采用patches捕获短程信息并聚合序列的全局表示。TPGN实现了$\mathcal{O}(\sqrt{L})$的理论复杂度,确保了其操作的高效性。在五个基准数据集上的实验结果表明,TPGN实现了最先进(SOTA)的性能和高效率,进一步证实了PGN作为RNN新继任者在长程时间序列预测中的有效性。代码可在以下仓库获取:\url{https://github.com/Water2sea/TPGN}。

英文摘要:

Due to the recurrent structure of RNN, the long information propagation path poses limitations in capturing long-term dependencies, gradient explosion/vanishing issues, and inefficient sequential execution. Based on this, we propose a novel paradigm called Parallel Gated Network (PGN) as the new successor to RNN. PGN directly captures information from previous time steps through the designed Historical Information Extraction (HIE) layer and leverages gated mechanisms to select and fuse it with the current time step information. This reduces the information propagation path to $\mathcal{O}(1)$, effectively addressing the limitations of RNN. To enhance PGN's performance in long-range time series forecasting tasks, we propose a novel temporal modeling framework called Temporal PGN (TPGN). TPGN incorporates two branches to comprehensively capture the semantic information of time series. One branch utilizes PGN to capture long-term periodic patterns while preserving their local characteristics. The other branch employs patches to capture short-term information and aggregate the global representation of the series. TPGN achieves a theoretical complexity of $\mathcal{O}(\sqrt{L})$, ensuring efficiency in its operations. Experimental results on five benchmark datasets demonstrate the state-of-the-art (SOTA) performance and high efficiency of TPGN, further confirming the effectiveness of PGN as the new successor to RNN in long-range time series forecasting. The code is available in this repository: \url{https://github.com/Water2sea/TPGN}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑