关于等变正则化在鲁棒在线持续学习中的有效性研究
On the Effectiveness of Equivariant Regularization for Robust Online Continual Learning
AI总结：
针对对比自监督学习在仅允许单遍数据的在线持续学习中样本效率低、效果有限的问题，提出结合等变正则化的自监督持续学习方法CLER，首次融合等变知识与持续学习，可集成到现有方法中，并通过消融实验揭示了等变任务对网络信息流及持续学习动态的影响。
摘要：
人类能够渐进式学习，而神经网络会灾难性地遗忘先前习得的信息。持续学习（CL）方法旨在通过在训练过程中促进知识向先前任务（后向迁移）和未来任务（前向迁移）的迁移，来缩小这一差距。\n近期研究表明，自监督学习能够生成可泛化到多种下游任务的通用模型。然而，对比自监督学习（CSSL）这一流行的自监督技术，在在线持续学习（OCL）中的效果有限。OCL仅允许对输入数据集进行一次遍历，而CSSL的样本效率较低，阻碍了其在输入数据流上的应用。\n在本研究中，我们提出了基于等变正则化的持续学习方法（CLER），这是一种利用等变任务实现自监督、规避CSSL局限性的OCL方法。我们的方法是首次将等变知识与持续学习相结合的尝试，且可轻松集成到现有OCL方法中。大量消融实验揭示了等变前置任务如何影响网络的信息流，以及其对持续学习动态的影响。
英文摘要：
Humans can learn incrementally, whereas neural networks forget previously acquired information catastrophically. Continual Learning (CL) approaches seek to bridge this gap by facilitating the transfer of knowledge to both previous tasks (backward transfer) and future ones (forward transfer) during training. Recent research has shown that self-supervision can produce versatile models that can generalize well to diverse downstream tasks. However, contrastive self-supervised learning (CSSL), a popular self-supervision technique, has limited effectiveness in online CL (OCL). OCL only permits one iteration of the input dataset, and CSSL's low sample efficiency hinders its use on the input data-stream. In this work, we propose Continual Learning via Equivariant Regularization (CLER), an OCL approach that leverages equivariant tasks for self-supervision, avoiding CSSL's limitations. Our method represents the first attempt at combining equivariant knowledge with CL and can be easily integrated with existing OCL methods. Extensive ablations shed light on how equivariant pretext tasks affect the network's information flow and its impact on CL dynamics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
