arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.03545cs.LG
收藏

在线特征更新改进在线(广义)标签偏移自适应

Online Feature Updates Improve Online (Generalized) Label Shift Adaptation

UC San Diego(加利福尼亚大学圣迭戈分校);Cornell University(康奈尔大学);University of Oxford(牛津大学);Google DeepMind(谷歌DeepMind)

Ruihan Wu, Siddhartha Datta, Yi Su, Dheeraj Baby, Yu-Xiang Wang, Kilian Q. Weinberger

AI总结:

针对在线场景下带标签缺失的标签偏移问题,提出结合自监督学习在线更新特征的OLS-OFU方法,理论上保持相近在线遗憾收敛性,实验效果较现有方法有显著提升。

摘要:

本文针对在线场景下存在标签缺失的普遍标签偏移问题展开研究,该场景中数据分布随时间变化,且及时获取标签难度较大。现有方法主要聚焦于调整或更新预训练分类器的最后一层,本文则挖掘了测试阶段利用无标签数据增强特征表示的未被开发的潜力。我们提出的新方法——带在线特征更新的在线标签偏移自适应(OLS-OFU),借助自监督学习优化特征提取流程,进而提升预测模型性能。通过精心设计算法,理论上OLS-OFU在考虑特征改进的同时,保持了与现有文献结果相近的在线遗憾收敛性。实验结果表明,该方法相比现有方法取得了显著提升,提升幅度与现有方法相对于基线(即未进行分布偏移自适应)的提升幅度相当。

英文摘要:

This paper addresses the prevalent issue of label shift in an online setting with missing labels, where data distributions change over time and obtaining timely labels is challenging. While existing methods primarily focus on adjusting or updating the final layer of a pre-trained classifier, we explore the untapped potential of enhancing feature representations using unlabeled data at test-time. Our novel method, Online Label Shift adaptation with Online Feature Updates (OLS-OFU), leverages self-supervised learning to refine the feature extraction process, thereby improving the prediction model. By carefully designing the algorithm, theoretically OLS-OFU maintains the similar online regret convergence to the results in the literature while taking the improved features into account. Empirically, it achieves substantial improvements over existing methods, which is as significant as the gains existing methods have over the baseline (i.e., without distribution shift adaptations).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑