arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.00655cs.LG
收藏

PatchMixer:一种用于长期时间序列预测的Patch混合架构

PatchMixer: A Patch-Mixing Architecture for Long-Term Time Series Forecasting

Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))

Zeying Gong, Yujin Tang, Junwei Liang

AI总结:

针对Transformer自注意力机制丢失时间信息的问题,提出基于CNN的PatchMixer模型,利用深度可分离卷积和双重预测头在单尺度架构下提取局部与全局特征,在七个基准上实现SOTA性能且速度提升2-3倍。

摘要:

尽管Transformer近年来一直是时间序列预测任务的主导架构,但一个基本挑战依然存在:Transformer内的置换不变自注意力机制会导致时间信息丢失。为了应对这些挑战,我们提出了一种基于CNN的新模型PatchMixer。它引入了置换可变卷积结构来保留时间信息。与该领域通常采用多尺度或众多分支的传统CNN不同,我们的方法仅依赖深度可分离卷积。这使我们能够使用单尺度架构提取局部特征和全局相关性。此外,我们采用了包含线性和非线性组件的双重预测头,以更好地建模未来曲线趋势和细节。我们在七个时间序列预测基准上的实验结果表明,与最先进方法和性能最佳的CNN相比,PatchMixer分别取得了3.9%和21.2%的相对提升,同时比最先进的方法快2-3倍。

英文摘要:

Although the Transformer has been the dominant architecture for time series forecasting tasks in recent years, a fundamental challenge remains: the permutation-invariant self-attention mechanism within Transformers leads to a loss of temporal information. To tackle these challenges, we propose PatchMixer, a novel CNN-based model. It introduces a permutation-variant convolutional structure to preserve temporal information. Diverging from conventional CNNs in this field, which often employ multiple scales or numerous branches, our method relies exclusively on depthwise separable convolutions. This allows us to extract both local features and global correlations using a single-scale architecture. Furthermore, we employ dual forecasting heads encompassing linear and nonlinear components to better model future curve trends and details. Our experimental results on seven time-series forecasting benchmarks indicate that compared with the state-of-the-art method and the best-performing CNN, PatchMixer yields $3.9\%$ and $21.2\%$ relative improvements, respectively, while being 2-3x faster than the most advanced method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑