Branch-Tuning：平衡稳定性与可塑性以实现持续自监督学习
Branch-Tuning: Balancing Stability and Plasticity for Continual Self-Supervised Learning
AI总结：
本文提出Branch-tuning方法，通过分析批归一化与卷积层的作用，在不修改原方法或保留旧数据的前提下平衡持续自监督学习中的稳定性与可塑性。
摘要：
自监督学习（SSL）已成为从海量无标签数据中提取通用表示的有效范式。然而，随着现实应用不断整合新内容，SSL极高的计算与资源需求使得持续学习成为必然，而非完全重新训练。这在适应新信息时带来了平衡稳定性与可塑性的挑战。本文采用Centered Kernel Alignment定量分析模型的稳定性与可塑性，揭示了批归一化层对稳定性的关键作用以及卷积层对可塑性的关键作用。受此启发，我们提出了Branch-tuning，一种高效且直接的方法，能够在持续SSL中实现稳定性与可塑性的平衡。Branch-tuning包含分支扩展与压缩，可轻松应用于各种SSL方法，无需修改原始方法、保留旧数据或模型。我们通过在多个基准数据集上的增量实验验证了该方法，证明了其在现实场景中的有效性与实用价值。我们希望本研究能为未来持续自监督学习研究提供新见解。代码将公开发布。
英文摘要：
Self-supervised learning (SSL) has emerged as an effective paradigm for deriving general representations from vast amounts of unlabeled data. However, as real-world applications continually integrate new content, the high computational and resource demands of SSL necessitate continual learning rather than complete retraining. This poses a challenge in striking a balance between stability and plasticity when adapting to new information. In this paper, we employ Centered Kernel Alignment for quantitatively analyzing model stability and plasticity, revealing the critical roles of batch normalization layers for stability and convolutional layers for plasticity. Motivated by this, we propose Branch-tuning, an efficient and straightforward method that achieves a balance between stability and plasticity in continual SSL. Branch-tuning consists of branch expansion and compression, and can be easily applied to various SSL methods without the need of modifying the original methods, retaining old data or models. We validate our method through incremental experiments on various benchmark datasets, demonstrating its effectiveness and practical value in real-world scenarios. We hope our work offers new insights for future continual self-supervised learning research. The code will be made publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
