arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.13208cs.LG
收藏

每个预测需要多少计算量?两种时间序列基础模型中的结构化块跳过

How Much Computation Does Every Forecast Need? Structured Block Skipping in Two Time-Series Foundation Models

Polytechnique Montréal(蒙特利尔理工学院)

Max Bourgeat

AI总结:

该研究针对时间序列基础模型推理计算冗余问题,在Chronos-Bolt和TimesFM 2.5中测试结构化块跳过,中等稀疏度下可降低延迟且预测质量变化小,但激进跳过会损害部分数据集性能,尚未形成可部署的动态推理方法。

摘要:

时间序列基础模型对所有输入使用相同的网络深度,尽管预测窗口的结构和难度存在差异。我们在两种预训练模型Chronos-Bolt和TimesFM 2.5中测试推理时是否可以移除整个Transformer块。我们的旁路操作避免了原始块的计算,且当启用所有块时能精确保留稠密模型。掩码仅在校准窗口上选择,并在ETTh1、Traffic、Weather和Exchange的1088个保留窗口上评估。在中等稀疏度下,Chronos-Bolt使用7/11候选块、TimesFM使用16/20块时,各数据集的平均相对预测质量几乎无变化,同时将A100批次1的中位数延迟分别降低11.2%和13.4%。更激进的TimesFM块跳过进一步加速推理,但会严重损害Traffic和Weather数据集的性能。在两种模型中,不同等预算掩码在不同窗口上表现最佳,且回顾式先知方法在每个数据集上均显著优于最佳静态掩码。不过,特定数据集的掩码迁移性不一致,Chronos的表示不支持可靠的未见域路由。这些结果确立了跨架构结构化冗余和输入依赖的机会,但尚未形成可部署的动态推理方法。

英文摘要:

Time-series foundation models use the same network depth for every input, although forecasting windows differ in structure and difficulty. We test whether whole Transformer blocks can be removed at inference time in two pretrained models: Chronos-Bolt and TimesFM 2.5. Our bypasses avoid the original block computation and preserve the dense model exactly when all blocks are enabled. Masks are selected only on calibration windows and evaluated on 1,088 held-out windows from ETTh1, Traffic, Weather, and Exchange. At moderate sparsity, Chronos-Bolt with 7/11 candidate blocks and TimesFM with 16/20 blocks show little change in mean relative forecasting quality across datasets while reducing median A100 batch-1 latency by 11.2% and 13.4%. More aggressive TimesFM skipping accelerates inference further but substantially harms Traffic and Weather. Across both models, different equal-budget masks win on different windows, and a retrospective oracle significantly improves over the best static mask on every dataset. However, dataset-specific masks transfer inconsistently, and Chronos representations do not support reliable unseen-domain routing. These results establish cross-architecture structured redundancy and input-dependent opportunity, but not a deployable dynamic inference method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑