arXiv 2211.122922023-07-27cs.CV

基于门控类注意力和级联特征漂移补偿的Vision Transformers无样本持续学习

Exemplar-free Continual Learning of Vision Transformers via Gated Class-Attention and Cascaded Feature Drift Compensation

University of Pavia（帕维亚大学）；Computer Vision Center（计算机视觉中心）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）；University of Florence（佛罗伦萨大学）

Marco Cotogni, Fei Yang, Claudio Cusano, Andrew D. Bagdanov, Joost van de Weijer

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对ViT无样本持续学习中的灾难性遗忘问题，提出门控类注意力与级联特征漂移补偿方法，在无需任务ID和样本回放的情况下平衡可塑性与记忆，取得优于基于排练方法的竞争力表现。

摘要：

我们提出了一种用于ViTs无样本类增量训练的新方法。无样本持续学习的主要挑战是在不导致对先前学习任务的灾难性遗忘的情况下，保持学习器的可塑性。这通常通过样本回放来实现，该机制可以帮助将先前任务的分类器重新校准到学习新任务时发生的特征漂移上。然而，样本回放是以保留先前任务的样本为代价的，这对于许多应用而言可能是不可行的。为了解决ViT持续训练的问题，我们首先提出了门控类注意力，以最小化最后一个ViT transformer块中的漂移。这种基于掩码的门控应用于最后一个transformer块的类注意力机制，并强烈调节对先前任务至关重要的权重。重要的是，门控类注意力在推理期间不需要任务ID，这使其与其他参数隔离方法区别开来。其次，我们提出了一种新的特征漂移补偿方法，以在学习新任务时适应主干网络中的特征漂移。门控类注意力和级联特征漂移补偿的结合允许对新任务的可塑性，同时限制了对先前任务的遗忘。在CIFAR-100、Tiny-ImageNet和ImageNet100上进行的广泛实验表明，与基于排练的ViT方法相比，我们的无样本方法获得了具有竞争力的结果。

英文摘要：

We propose a new method for exemplar-free class incremental training of ViTs. The main challenge of exemplar-free continual learning is maintaining plasticity of the learner without causing catastrophic forgetting of previously learned tasks. This is often achieved via exemplar replay which can help recalibrate previous task classifiers to the feature drift which occurs when learning new tasks. Exemplar replay, however, comes at the cost of retaining samples from previous tasks which for many applications may not be possible. To address the problem of continual ViT training, we first propose gated class-attention to minimize the drift in the final ViT transformer block. This mask-based gating is applied to class-attention mechanism of the last transformer block and strongly regulates the weights crucial for previous tasks. Importantly, gated class-attention does not require the task-ID during inference, which distinguishes it from other parameter isolation methods. Secondly, we propose a new method of feature drift compensation that accommodates feature drift in the backbone when learning new tasks. The combination of gated class-attention and cascaded feature drift compensation allows for plasticity towards new tasks while limiting forgetting of previous ones. Extensive experiments performed on CIFAR-100, Tiny-ImageNet and ImageNet100 demonstrate that our exemplar-free method obtains competitive results when compared to rehearsal based ViT methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。