arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.05908cs.LG
收藏

非平稳学习中的实例条件衰减时间尺度

Instance-Conditional Timescales of Decay for Non-Stationary Learning

Google Research India(谷歌研究院印度)

Nishant Jain, Pradeep Shenoy

AI总结:

针对非平稳学习中的缓慢概念漂移,提出优化驱动的实例重要性平衡方法,利用多时间尺度衰减和嵌套优化学习评分器,在大型真实数据集上显著提升准确率并超越现有方法。

摘要:

缓慢的概念漂移是实际机器学习系统中普遍存在但研究不足的问题。在此类设置中,尽管近期数据更能指示未来数据,但天真地优先考虑近期实例可能会丢失来自过去的宝贵信息。我们提出了一种优化驱动的方法,用于在大型训练窗口内平衡实例重要性。首先,我们使用多个衰减时间尺度的混合来建模实例相关性,从而捕捉丰富的时间趋势。其次,我们学习一个辅助评分模型,该模型根据实例本身恢复适当的时间尺度混合。最后,我们提出了一个用于学习评分器的嵌套优化目标,通过该目标,它最大化所学模型的前向迁移。在9年期间3900万张照片的大规模真实世界数据集上的实验表明,与其他稳健学习基线相比,准确率相对提升高达15%。我们在两个真实世界数据集的集合上复制了我们的增益,用于非平稳学习,并将我们的工作扩展到持续学习设置,在这些设置中,我们也大幅超越了最先进的方法。

英文摘要:

Slow concept drift is a ubiquitous, yet under-studied problem in practical machine learning systems. In such settings, although recent data is more indicative of future data, naively prioritizing recent instances runs the risk of losing valuable information from the past. We propose an optimization-driven approach towards balancing instance importance over large training windows. First, we model instance relevance using a mixture of multiple timescales of decay, allowing us to capture rich temporal trends. Second, we learn an auxiliary scorer model that recovers the appropriate mixture of timescales as a function of the instance itself. Finally, we propose a nested optimization objective for learning the scorer, by which it maximizes forward transfer for the learned model. Experiments on a large real-world dataset of 39M photos over a 9 year period show upto 15% relative gains in accuracy compared to other robust learning baselines. We replicate our gains on two collections of real-world datasets for non-stationary learning, and extend our work to continual learning settings where, too, we beat SOTA methods by large margins.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑