arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.33822cs.LGcs.AI
收藏

学习共享内容:面向时间序列预测的专家专业化

Learning What to Share: Expert Specialization for Time-Series Forecasting

Technion – Israel Institute of Technology(以色列理工学院)

Bashar Khoury, Shadi Omari, Assaf Schuster

AI总结:

该研究针对多变量预测的跨通道共享与专业化问题,提出即插即用的持久闭式路由方法,在11个数据集上实现了3.86-14.25%的均方误差降低,且参数更少。

摘要:

多变量预测的核心挑战是确定跨通道共享什么内容以及在何处进行专业化。混合专家模型提供了灵活的共享方式,但目前尚不清楚其预测性能提升有多少来自增加的容量或专家分配。通过在11个数据集和3种骨干网络上进行受控比较,我们表明,信息丰富的通道分配在匹配共享容量和随机分配的基础上进一步提升了预测效果。在冻结的Transformer骨干网络上,持久的通道特定混合方法平均表现优于依赖输入的路由方法。为了研究专业化何时以及在多大程度上有帮助,我们引入了GalaxyTS,这是一个具有精确共享和通道特定预测参考的受控合成数据集。它揭示了持久通道混合如何利用短输入窗口所掩盖的预测差异,并测量了在匹配共享修正之上的增益。这些见解催生了持久闭式路由(PCR),这是一种即插即用方法,它交替进行通道特定混合权重的精确条件估计,以及围绕冻结预训练预测器的低秩残差专家的基于梯度的训练。在4个预测 horizon 上取平均值,PCR-K4相对于3种冻结骨干网络,实现了数据集平衡均方误差降低3.86-14.25%。在匹配比较中,PCR平均优于竞争的专业化方法,同时需要的可训练参数显著更少。

英文摘要:

A central challenge in multivariate forecasting is deciding what to share across channels and where to specialize. Mixture-of-experts models offer flexible sharing, yet it remains unclear how much of their forecasting gains comes from added capacity or expert allocation. Through controlled comparisons across eleven datasets and three backbones, we show that informative channel assignments improve forecasting beyond matched shared capacity and random allocation. On frozen transformer backbones, persistent channel-specific mixtures outperform input-dependent routing on average. To examine when and how much specialization helps, we introduce GalaxyTS, a controlled synthetic dataset with exact shared and channel-specific forecasting references. It reveals how persistent channel mixtures exploit predictive differences concealed by short input windows and measures gains over matched shared corrections. These insights motivate Persistent Closed-Form Routing (PCR), a plug-and-play method that alternates exact conditional estimation of channel-specific mixture weights with gradient-based training of low-rank residual experts around a frozen pretrained forecaster. Averaged over four forecasting horizons, PCR-K4 achieves dataset-balanced mean squared error reductions of 3.86-14.25% relative to the three frozen backbones. In matched comparisons, PCR outperforms competing specialization methods on average while requiring substantially fewer trainable parameters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑