arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.08672cs.LGcs.AIstat.ME
收藏

时间分布偏移下的模型评估与选择

Model Assessment and Selection under Temporal Distribution Shift

Columbia University(哥伦比亚大学)

Elise Han, Chengpiao Huang, Kaizheng Wang

AI总结:

针对变化环境中的模型评估与选择问题,提出自适应滚动窗口方法估计泛化误差,并结合单败淘汰赛实现近优模型选择,理论与实验证明其对数据非平稳性具有自适应性。

摘要:

我们通过综合当前时间段和历史时期的数据集,研究变化环境中的模型评估与选择。为应对未知且可能任意的时间分布偏移,我们开发了一种自适应滚动窗口方法来估计给定模型的泛化误差。该策略还便于通过估计任意两个候选模型泛化误差之差来比较它们。我们进一步将两两比较整合到单败淘汰赛中,实现从候选集合中进行近优模型选择。理论分析与数值实验表明,所提方法对数据非平稳性具有自适应性。

英文摘要:

We investigate model assessment and selection in a changing environment, by synthesizing datasets from both the current time period and historical epochs. To tackle unknown and potentially arbitrary temporal distribution shift, we develop an adaptive rolling window approach to estimate the generalization error of a given model. This strategy also facilitates the comparison between any two candidate models by estimating the difference of their generalization errors. We further integrate pairwise comparisons into a single-elimination tournament, achieving near-optimal model selection from a collection of candidates. Theoretical analyses and numerical experiments demonstrate the adaptivity of our proposed methods to the non-stationarity in data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑