arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.18367cs.LG
收藏

时间序列基础模型基准是否隐藏了依赖于状态的失败?来自交通速度预测的证据

Do Time Series Foundation Model Benchmarks Hide Regime-Dependent Failures? Evidence from Traffic Speed Forecasting

University of California, Berkeley(加州大学伯克利分校);Duke University(杜克大学);National University of Singapore(新加坡国立大学);Northeastern University(东北大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Southern Methodist University(南卫理公会大学)

Yingshuo Wang, Xian Sun, Lingdong Kong, Wei Gao, Yanhang Li, Zhichao Fan, Zexin Zhuang

AI总结:

本文提出状态分层评估方法,发现时间序列基础模型在交通状态转换时准确率和预测区间覆盖率显著下降,并提出了双峰混合增强方法以改善转换状态覆盖。

摘要:

标准基准使用聚合指标评估时间序列基础模型(TSFMs),但这可能掩盖关键运行状态下的严重失败。我们引入了状态分层评估,并将其应用于两个标准交通速度基准上的三个TSFMs。交通在自由流和拥堵状态之间表现出突然的状态切换,在转换期间产生双峰速度分布。当我们按交通状态分层时,准确率和预测区间覆盖率在转换期间急剧下降:转换状态的MAE达到11 mph(而总体为3 mph),90%预测区间的经验覆盖率低至55%。这些失败在聚合指标中不可见,因为自由流观测主导了样本。一个简单的历史条件基线(从每个传感器的训练分布中采样)实现了比任何TSFM更好的转换覆盖率,但总体准确率差得多。我们提出了双峰混合增强(BMA),一种后处理方法,将TSFM预测与历史分布知识相结合,在保持TSFM准确率的同时接近历史基线的转换覆盖率。我们的结果表明,TSFM基准应纳入状态感知评估,以揭示聚合指标隐藏的失败。

英文摘要:

Standard benchmarks evaluate time series foundation models (TSFMs) using aggregate metrics, but these can mask severe failures in critical operating regimes. We introduce regime-stratified evaluation and apply it to three TSFMs on two standard traffic speed benchmarks. Traffic exhibits abrupt regime switching between free-flow and congested states, producing bimodal speed distributions during transitions. When we stratify by traffic regime, both accuracy and prediction-interval coverage degrade sharply during transitions: transition-regime MAE reaches 11 mph (versus 3 mph overall), and empirical coverage of 90% prediction intervals drops as low as 55%. These failures are invisible in aggregate metrics because free-flow observations dominate the sample. A simple historical conditional baseline (sampling from per-sensor training distributions) achieves better transition coverage than any TSFM, but has far worse overall accuracy. We propose bimodal mixture augmentation (BMA), a post-hoc method that combines TSFM forecasts with historical distributional knowledge, approaching the historical baseline's transition coverage while preserving the TSFM's accuracy. Our results suggest that TSFM benchmarks should incorporate regime-aware evaluation to surface failures that aggregate metrics hide.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑