arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2607.06607cs.LGcs.AI

NEST:通过面向机制的专家混合方法应对数据集级别的分布变化

NEST: Tackling Dataset-Level Distribution Shifts via Regime-Oriented Mixture-of-Experts

Beihang University(北京航空航天大学)

Lanhao Li, Bingshu Xie, Lijun Sun, Xin Xue, Haoyi Zhou, Jianxin Li

AI总结:

针对复杂系统中数据集级分布变化影响长期预测问题,提出NEST框架,通过两阶段密集MoE架构,经无监督聚类和面向机制的路由器机制建模重组演变结构,在多基准测试中性能达最优。

摘要:

复杂系统中的准确长期预测常受数据集级分布变化影响,现有方法多关注局部时间变化,未明确建模数据集由不同运行机制组成的全局结构挑战。本文提出NEST框架,通过两阶段密集MoE架构对这些演变结构进行建模和重组。首先在原则性矩熵空间中通过无监督聚类将数据集划分为不同运行机制以促进结构专业化,引入面向机制的路由器机制生成初始专家权重并经几何调制优化。各专家作为专门内核捕捉特定机制动态。在多种基准测试中NEST性能达最优。

英文摘要:

Accurate long-term forecasting in complex systems is frequently compromised by dataset-level distribution shifts, where diverse underlying behavioral modes and evolving system states drive the dynamic multivariate time-series. While existing methods predominantly focus on local temporal shifts, they fail to explicitly model the global structural challenge where datasets are composites of distinct operational regimes. In this paper, we propose NEST, a specialized framework designed to model and recompose these evolving structures through a two-phase dense MoE architecture. NEST first facilitates structural specialization by partitioning the dataset into distinct operational regimes through unsupervised clustering in a principled moment-entropy space. We introduce a regime-oriented router mechanism that generates initial expert weights based on temporal content, subsequently refined through geometric modulation to regime centroids. Crucially, rather than acting as monolithic predictors, individual experts function as specialized kernels that capture regime-specific dynamics by evolving unique variate-attention patterns. Extensive evaluations on diverse benchmarks, including heterogeneous network traffic and physical phenomena, demonstrate that NEST consistently achieves state-of-the-art performance. Our code and datasets are available at https://github.com/Aaralshin/NEST

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑