arXiv 2207.078272025-03-25cs.LGcs.CV

缓解数据冗余以重振基于Transformer的长期时间序列预测系统

Mitigating Data Redundancy to Revitalize Transformer-based Long-Term Time Series Forecasting System

Stanford University（斯坦福大学）；University of Technology Sydney（悉尼科技大学）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61）；University of New South Wales（新南威尔士大学）；University of Science and Technology of China（中国科学技术大学）

Mingjie Li, Rui Liu, Guangsi Shi, Mingfei Han, Changling Li, Lina Yao, Xiaojun Chang, Ling Chen

备注：: ACM TIST

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对Transformer在长期时间序列预测中因数据冗余导致的过拟合问题，提出CLMFormer框架，通过课程噪声和记忆解码器提升泛化，在六个基准上最高提升30%。

摘要：

长期时间序列预测（LTSF）是各种实际应用的基础，其中基于Transformer的模型因其捕捉长距离依赖的能力而成为主导框架。然而，在滚动预测设置中，这些模型常常因数据冗余而经历过度拟合，限制了其泛化能力，尤其在具有高度相似相邻数据的较长序列中这一问题尤为明显。在本工作中，我们引入了CLMFormer，一种通过课程学习和记忆驱动解码器来缓解冗余的新框架。具体来说，我们逐步向训练样本引入伯努利噪声，这有效打破了相邻数据点之间的高度相似性。这种课程驱动的噪声引入通过提供更多样化和有代表性的训练数据来辅助记忆驱动解码器，增强了解码器对时间序列数据中季节趋势和依赖关系的建模能力。为了进一步提高预测精度，我们引入了记忆驱动解码器。该组件使模型能够捕捉时间序列数据中的季节趋势和依赖关系，并利用时间关系来促进预测过程。在六个真实世界LTSF基准上的广泛实验表明，CLMFormer持续将基于Transformer的模型性能提升高达30%，证明了其在长时域预测中的有效性。

英文摘要：

Long-term time-series forecasting (LTSF) is fundamental to various real-world applications, where Transformer-based models have become the dominant framework due to their ability to capture long-range dependencies. However, these models often experience overfitting due to data redundancy in rolling forecasting settings, limiting their generalization ability particularly evident in longer sequences with highly similar adjacent data. In this work, we introduce CLMFormer, a novel framework that mitigates redundancy through curriculum learning and a memory-driven decoder. Specifically, we progressively introduce Bernoulli noise to the training samples, which effectively breaks the high similarity between adjacent data points. This curriculum-driven noise introduction aids the memory-driven decoder by supplying more diverse and representative training data, enhancing the decoder's ability to model seasonal tendencies and dependencies in the time-series data. To further enhance forecasting accuracy, we introduce a memory-driven decoder. This component enables the model to capture seasonal tendencies and dependencies in the time-series data and leverages temporal relationships to facilitate the forecasting process. Extensive experiments on six real-world LTSF benchmarks show that CLMFormer consistently improves Transformer-based models by up to 30%, demonstrating its effectiveness in long-horizon forecasting.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。