arXiv 2603.185962026-03-26cs.LGcs.AIcs.CV

在持续学习中正确实现弹性权重整合

Elastic Weight Consolidation Done Right for Continual Learning

School of Artificial Intelligence, Sun Yat-sen University, China（中山大学人工智能学院，中国）；Key Laboratory of Intelligent Assessment Technology for Sustainable Tourism, Ministry of Culture and Tourism, Sun Yat-sen University（文化旅游可持续发展智能评估技术重点实验室，文化部，中山大学）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China（机器智能与高级计算重点实验室，教育部，中国）

Xuan Liu, Xiaobin Chang

备注：: Accepted to CVPR 2026
发表信息：: IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文分析了EWC中权重重要性估计的问题，提出Logits Reversal操作改进EWC，通过防止梯度消失和冗余保护提升持续学习性能。

摘要：

持续学习（CL）中的权重正则化方法通过评估和惩罚重要模型权重的变化来缓解灾难性遗忘。弹性权重整合（EWC）是该框架中的基础方法，基于梯度估计权重重要性。然而，它在某些场景下表现不佳。本文从梯度角度系统分析了EWC的重要性估计，发现其依赖Fisher信息矩阵（FIM）导致梯度消失和重要性估计不准确。此外，Memory Aware Synapses（MAS）对无关先前任务的参数施加不必要的约束，称为冗余保护。因此，EWC及其变体在估计权重重要性时存在根本性偏差，导致性能不佳。为解决这些问题，我们提出Logits Reversal（LR）操作，通过反转FIM计算中的logit值防止梯度消失和冗余保护。在各种CL任务和数据集上的实验表明，所提方法显著优于现有EWC及其变体。因此，我们称之为EWC Done Right（EWC-DR）。代码可在https://github.com/scarlet0703/EWC-DR获取。

英文摘要：

Weight regularization methods in continual learning (CL) alleviate catastrophic forgetting by assessing and penalizing changes to important model weights. Elastic Weight Consolidation (EWC) is a foundational and widely used approach within this framework that estimates weight importance based on gradients. However, it has consistently shown suboptimal performance. In this paper, we conduct a systematic analysis of importance estimation in EWC from a gradient-based perspective. For the first time, we find that EWC's reliance on the Fisher Information Matrix (FIM) results in gradient vanishing and inaccurate importance estimation in certain scenarios. Our analysis also reveals that Memory Aware Synapses (MAS), a variant of EWC, imposes unnecessary constraints on parameters irrelevant to prior tasks, termed the redundant protection. Consequently, both EWC and its variants exhibit fundamental misalignments in estimating weight importance, leading to inferior performance. To tackle these issues, we propose the Logits Reversal (LR) operation, a simple yet effective modification that rectifies EWC's importance estimation. Specifically, reversing the logit values during the calculation of FIM can effectively prevent both gradient vanishing and redundant protection. Extensive experiments across various CL tasks and datasets show that the proposed method significantly outperforms existing EWC and its variants. Therefore, we refer to it as EWC Done Right (EWC-DR). Code is available at https://github.com/scarlet0703/EWC-DR.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。