MoHETS：基于混合异质专家的长期时间序列预测
MoHETS: Long-term Time Series Forecasting with Mixture-of-Heterogeneous-Experts
AI总结：
本文提出MoHETS，一种仅含编码器的Transformer模型，通过整合稀疏混合异质专家层，有效捕捉多尺度时间序列结构，提升非平稳动态的鲁棒性，并在多个基准测试中取得最佳性能，平均MSE降低12%。
摘要：
现实中的多变量时间序列表现出复杂的多尺度结构，包括全球趋势、局部周期性和非平稳制度，这使得长期预测具有挑战性。尽管稀疏混合专家（MoE）方法提高了可扩展性和专业化，但通常依赖同质MLP专家，难以捕捉时间序列数据多样的时间动态。本文提出MoHETS，一种仅含编码器的Transformer，整合稀疏混合异质专家（MoHE）层。MoHE将时间片段路由到少量专家网络，结合共享深度卷积专家以保持序列连续性，以及通过路由傅里叶基专家捕捉片段级周期结构。MoHETS进一步通过交叉注意力机制整合外生信息以增强对非平稳动态的鲁棒性。最后，本文用轻量级卷积片段解码器替代参数密集的线性投影头，提高参数效率，减少训练不稳定性，并使单个模型能泛化到任意预测时间跨度。在七个多变量基准测试和多个时间跨度上验证，MoHETS在多个基准测试中均取得最佳性能，平均MSE比强近期基线降低12%，证明了其在长期预测中的有效异质专业化。
英文摘要：
Real-world multivariate time series can exhibit intricate multi-scale structures, including global trends, local periodicities, and non-stationary regimes, which makes long-horizon forecasting challenging. Although sparse Mixture-of-Experts (MoE) approaches improve scalability and specialization, they typically rely on homogeneous MLP experts that poorly capture the diverse temporal dynamics of time series data. We address these limitations with MoHETS, an encoder-only Transformer that integrates sparse Mixture-of-Heterogeneous-Experts (MoHE) layers. MoHE routes temporal patches to a small subset of expert networks, combining a shared depthwise-convolution expert for sequence-level continuity with routed Fourier-based experts for patch-level periodic structures. MoHETS further improves robustness to non-stationary dynamics by incorporating exogenous information via cross-attention over covariate patch embeddings. Finally, we replace parameter-heavy linear projection heads with a lightweight convolutional patch decoder, improving parameter efficiency, reducing training instability, and allowing a single model to generalize across arbitrary forecast horizons. We validate across seven multivariate benchmarks and multiple horizons, with MoHETS consistently achieving state-of-the-art performance, reducing the average MSE by $12\%$ compared to strong recent baselines, demonstrating effective heterogeneous specialization for long-term forecasting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
