检索机制在时间序列预测中超越长上下文扩展
Retrieval Mechanisms Surpass Long-Context Scaling in Time Series Forecasting
AI总结：
研究通过ETTh1基准测试发现，长上下文模型在时间序列预测中表现不佳，而检索增强预测方法在减少计算量的同时提升了预测精度。
摘要：
时间序列基础模型（TSFMs）在自然语言处理中借鉴了长上下文范式，假设增加历史输入能提升预测质量。但随机领域中，远距离历史常为高频噪声而非信号。本文通过ETTh1基准测试发现，连续上下文架构（包括PatchTST）的结果与假设矛盾：出现反比例定律，预测误差随上下文长度增加而上升。3000步窗口导致性能下降超过68%，表明注意力机制难以忽略无关历史波动。检索增强预测（RAFT）在固定720步窗口和选择性检索下，以0.379的均方误差超越长上下文配置和零样本基础模型（Chronos, Moirai），且计算需求远低。此外，检索步骤仅注入最相关的历史片段作为动态外生变量，为模型提供上下文引导的归纳偏差，无法仅凭原始序列自行构建。因此，未来基础模型需转向选择性检索架构。
英文摘要：
Time Series Foundation Models (TSFMs) have borrowed the long context paradigm from natural language processing under the premise that feeding more history into the model improves forecast quality. But in stochastic domains, distant history is often just high-frequency noise, not signal. Hence, the proposed work tests whether this premise actually holds by running continuous context architectures (PatchTST included) through the ETTh1 benchmark. The obtained results contradict the premise: an inverse scaling law shows up clearly, with forecasting error rising as context gets longer. A 3,000-step window causes performance to drop by over 68%, evidence that attention mechanisms are poor at ignoring irrelevant historical volatility. Retrieval-Augmented Forecasting (RAFT) is evaluated as an alternative. RAFT achieves a mean squared error (MSE) of 0.379 with a fixed 720-step window and selective retrieval, outperforming both long-context configurations and zero-shot foundation models (Chronos, Moirai) despite requiring far less computation. In addition, the retrieval step injects only the most relevant historical segments as dynamic exogenous variables, which gives the model a context-informed inductive bias it cannot build on its own from raw sequences. Therefore, foundation models going forward need to shift architecturally toward selective retrieval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
