EMTSF:时间序列预测中SOTA模型的非凡混合
EMTSF:Extraordinary Mixture of SOTA Models for Time Series Forecasting
AI总结:
针对时间序列预测中模型选择的争议,提出基于Transformer门控的MoE框架EMTSF,集成xLSTM、PatchTST等SOTA模型,在标准基准上超越现有所有方法。
摘要:
Transformer架构在自然语言处理领域的巨大成功,促使其被引入时间序列预测(TSF)领域,并展现出优越的性能。然而,近期一篇重要论文质疑了其有效性,证明一个简单的单层线性模型性能优于基于Transformer的模型。这一结论很快被一个更好的基于Transformer的模型PatchTST所推翻。最近,TimeLLM通过将大型语言模型(LLM)重新用于TSF领域,展示了更好的结果。然而,一篇后续论文对此提出挑战,证明移除LLM组件或将其替换为基本注意力层,实际上能获得更好的性能。预测中的一个挑战是,TSF数据更倾向于近期数据,且有时会受到不可预测事件的影响。基于TSF领域这些最新的见解,我们提出了一个强大的专家混合(MoE)框架。我们的方法结合了包括xLSTM、增强线性模型、PatchTST和minGRU等在内的最先进(SOTA)模型。这组互补且多样化的TSF模型被集成在一个基于Transformer的MoE门控网络中。我们提出的模型在标准基准上优于所有现有的TSF模型,甚至超越了基于MoE框架的最新方法。
英文摘要:
The immense success of the Transformer architecture in Natural Language Processing has led to its adoption in Time Se ries Forecasting (TSF), where superior performance has been shown. However, a recent important paper questioned their effectiveness by demonstrating that a simple single layer linear model outperforms Transformer-based models. This was soon shown to be not as valid, by a better transformer-based model termed PatchTST. More re cently, TimeLLM demonstrated even better results by repurposing a Large Language Model (LLM) for the TSF domain. Again, a follow up paper challenged this by demonstrating that removing the LLM component or replacing it with a basic attention layer in fact yields better performance. One of the challenges in forecasting is the fact that TSF data favors the more recent past, and is sometimes subject to unpredictable events. Based upon these recent insights in TSF, we propose a strong Mixture of Experts (MoE) framework. Our method combines the state-of-the-art (SOTA) models including xLSTM, en hanced Linear, PatchTST, and minGRU, among others. This set of complimentary and diverse models for TSF are integrated in a Trans former based MoE gating network. Our proposed model outperforms all existing TSF models on standard benchmarks, surpassing even the latest approaches based on MoE frameworks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。