发表机构
University of Washington（华盛顿大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文揭示自监控测试时自适应中，因预测误差重叠与依赖破坏可交换性假设，导致误报警且自适应掩盖变化，强调需检查假设并对比冻结模型。
AI 中文摘要
现代预测模型通常在部署后会被更新，以便对变化的数据作出响应。然而，这些更新也可能使预测变得更差，因此实际系统需要一个可靠的监控器来检测有害变化并触发保护机制。一种自然的设计是监控那些指导更新的同一预测误差。本文探讨了当监控和自适应使用相同反馈时，该监控器背后的统计保证是否仍然有效。我们在多步时间序列预测中研究这一问题。我们表明，重叠的目标和预测误差中的依赖性可能破坏该保证所需的一个关键假设。监控器可能在没有发生有害变化时发出警报，而其响应可能进一步损害预测质量。我们还发现，自适应可以对其自身的监控器隐藏持续的变化，而原始的冻结模型则保留更清晰的信号。这些结果揭示了自监控自适应中的一个基本失效模式。它们表明，可靠的部署需要检查监控器的假设，在现实反馈下将自适应与冻结模型进行比较，并限制每个保护性响应的效果。
英文摘要
Modern forecasting models are often updated after deployment so they can respond to changing data. These updates can also make predictions worse, so practical systems need a reliable monitor that can detect harmful changes and trigger protection. A natural design is to monitor the same prediction errors that guide the updates. This paper asks whether the statistical guarantee behind such a monitor remains valid when monitoring and adaptation use the same feedback. We study this question in multi-step time-series forecasting. We show that overlapping targets and dependence in forecast errors can break a key assumption required by the guarantee. The monitor may then raise alarms even when no harmful change has occurred, and its response can further damage prediction quality. We also find that adaptation can hide sustained changes from its own monitor, while the original frozen model retains a clearer signal. These results expose a basic failure mode in self-monitored adaptation. They show why reliable deployment requires checking the monitor's assumptions, comparing adaptation with the frozen model under realistic feedback, and limiting the effect of every protective response.
Comments49 pages, 9 figures
