arXiv 2406.173812026-02-01cs.LGcs.CV

回顾性特征估计用于持续学习

Retrospective Feature Estimation for Continual Learning

VinUni-Illinois Smart Health Center（VinUni-伊利诺伊州立健康中心）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；The Chinese University of Hong Kong（香港中文大学）；Simular Research（Simular研究公司）；University of Warwick（沃里克大学）

Nghia D. Nguyen, Hieu Trung Nguyen, Ang Li, Hoang Pham, Viet Anh Nguyen, Khoa D. Doan

备注：: Published in Transactions on Machine Learning Research with Featured Certification

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出回顾性特征估计方法，通过逆转特征变化来缓解持续学习中的灾难性遗忘问题。

摘要：

深度神经网络（DNNs）内在的持续学习变化数据流的能力是一个需求。然而，当前DNNs存在灾难性遗忘问题，影响对过去知识的回忆。为缓解此问题，现有持续学习（CL）方法通常保留示例进行重播、正则化学习或分配专用容量给新任务。本文探讨了持续学习中一个未探索的方向，称为回顾性特征估计（RFE）。RFE通过将当前训练DNN的特征向回映射到旧任务的特征空间，学习逆转特征变化，从而更容易进行预测。此回顾过程利用一系列小型特征映射网络，称为回溯模块。在多个持续学习基准测试中，包括CIFAR10、CIFAR100和Tiny ImageNet，实验证明了该新持续学习方向的有效性和潜力，相较于现有代表性持续学习方法，推动进一步研究回顾机制作为减轻持续学习中灾难性遗忘的原理性替代方案。代码可在：https://github.com/mail-research/retrospective-feature-estimation获取。

英文摘要：

The intrinsic capability to continuously learn a changing data stream is a desideratum of deep neural networks (DNNs). However, current DNNs suffer from catastrophic forgetting, which interferes with remembering past knowledge. To mitigate this issue, existing Continual Learning (CL) approaches often retain exemplars for replay, regularize learning, or allocate dedicated capacity for new tasks. This paper investigates an unexplored direction for CL called Retrospective Feature Estimation (RFE). RFE learns to reverse feature changes by aligning the features from the current trained DNN backward to the feature space of the old task, where performing predictions is easier. This retrospective process utilizes a chain of small feature mapping networks called retrospector modules. Empirical experiments on several CL benchmarks, including CIFAR10, CIFAR100, and Tiny ImageNet, demonstrate the effectiveness and potential of this novel CL direction compared to existing representative CL methods, motivating further research into retrospective mechanisms as a principled alternative for mitigating catastrophic forgetting in CL. Code is available at: https://github.com/mail-research/retrospective-feature-estimation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。