arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.01239cs.CVcs.LG
收藏

面向循环域偏移的持续学习在线蒸馏

Online Distillation with Continual Learning for Cyclic Domain Shifts

KAUST(阿卜杜拉国王科技大学)

Joachim Houyon, Anthony Cioppa, Yasir Ghunaim, Motasem Alfarra, Anaïs Halin, Maxim Henry, Bernard Ghanem, Marc Van Droogenbroeck

AI总结:

本文针对在线蒸馏在域偏移下的灾难性遗忘问题,将多种最先进持续学习方法融入师生在线蒸馏框架,并在循环域偏移场景验证其可提升鲁棒性与准确率。

摘要:

近年来,在线蒸馏已成为一种强大的技术,可借助速度较慢但准确的教师模型实时调整深度神经网络。然而,在线蒸馏的一个主要挑战是域偏移时的灾难性遗忘,即学生模型使用新域数据更新后会忘记先前学到的知识。本文中,我们提出利用持续学习方法的能力来降低域偏移影响,以解决该问题。具体而言,我们将若干最先进的持续学习方法集成到在线蒸馏场景中,并证明它们在减少灾难性遗忘方面的有效性。此外,我们针对循环域偏移情况对所提解决方案进行了详细分析。实验结果表明,该方法能有效提升在线蒸馏的鲁棒性和准确性,并在视频监控或自动驾驶等领域具有潜在应用价值。总体而言,我们的工作代表了在线蒸馏与持续学习领域的重要进展,并有可能对实际应用产生显著影响。

英文摘要:

In recent years, online distillation has emerged as a powerful technique for adapting real-time deep neural networks on the fly using a slow, but accurate teacher model. However, a major challenge in online distillation is catastrophic forgetting when the domain shifts, which occurs when the student model is updated with data from the new domain and forgets previously learned knowledge. In this paper, we propose a solution to this issue by leveraging the power of continual learning methods to reduce the impact of domain shifts. Specifically, we integrate several state-of-the-art continual learning methods in the context of online distillation and demonstrate their effectiveness in reducing catastrophic forgetting. Furthermore, we provide a detailed analysis of our proposed solution in the case of cyclic domain shifts. Our experimental results demonstrate the efficacy of our approach in improving the robustness and accuracy of online distillation, with potential applications in domains such as video surveillance or autonomous driving. Overall, our work represents an important step forward in the field of online distillation and continual learning, with the potential to significantly impact real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑