Seg-MoE：多分辨率分段式混合专家用于时间序列预测Transformer
Seg-MoE: Multi-Resolution Segment-wise Mixture-of-Experts for Time Series Forecasting Transformers
AI总结：
本文提出Seg-MoE，通过分段式混合专家机制提升时间序列预测的准确性，验证了分段路由在捕捉长期时间动态中的有效性。
摘要：
基于Transformer的模型在准确的时间序列预测中取得了显著进展，但这些架构在高效扩展和捕捉长期时间动态方面仍面临挑战。混合专家（MoE）层是解决自然语言处理中扩展问题的已知解决方案。然而，现有时间序列预测中的MoE方法依赖于token级别的路由机制，可能无法有效利用时间数据的自然局部性和连续性。本文引入Seg-MoE，一种稀疏MoE设计，通过连续时间步段进行路由和处理，而非独立专家决策。token段允许每个专家直接建模段内交互，自然对齐固有的时间模式。我们将Seg-MoE层集成到时间序列Transformer中，并在多个多变量长期预测基准上进行评估。Seg-MoE在几乎所有预测时间范围内均取得最先进的预测准确性，优于密集Transformer和先前基于token的MoE模型。全面的消融研究证实，段级路由是推动这些增益的关键因素。我们的结果表明，将MoE路由粒度与时间序列的固有结构对齐，提供了一种强大的、此前未被探索的归纳偏差，为序列数据建模中的条件稀疏架构开辟了新途径。
英文摘要：
Transformer-based models have recently made significant advances in accurate time-series forecasting, but even these architectures struggle to scale efficiently while capturing long-term temporal dynamics. Mixture-of-Experts (MoE) layers are a proven solution to scaling problems in natural language processing. However, existing MoE approaches for time-series forecasting rely on token-wise routing mechanisms, which may fail to exploit the natural locality and continuity of temporal data. In this work, we introduce Seg-MoE, a sparse MoE design that routes and processes contiguous time-step segments rather than making independent expert decisions. Token segments allow each expert to model intra-segment interactions directly, naturally aligning with inherent temporal patterns. We integrate Seg-MoE layers into a time-series Transformer and evaluate it on multiple multivariate long-term forecasting benchmarks. Seg-MoE consistently achieves state-of-the-art forecasting accuracy across almost all prediction horizons, outperforming both dense Transformers and prior token-wise MoE models. Comprehensive ablation studies confirm that segment-level routing is the key factor driving these gains. Our results show that aligning the MoE routing granularity with the inherent structure of time series provides a powerful, yet previously underexplored, inductive bias, opening new avenues for conditionally sparse architectures in sequential data modeling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
