MIND：多任务增量网络蒸馏
MIND: Multi-Task Incremental Network Distillation
AI总结：
MIND是一种参数隔离方法，通过两种蒸馏程序和批归一化层优化，在无重放类增量学习（CIFAR-100/10提升约+6%，TinyImageNet/10提升约+10%）和域增量场景（约+40%）中达到最先进性能。
摘要：
近年来，生成动态数据流的普及设备激增，凸显了学习系统持续适应数据分布变化的需求。为应对这一挑战，研究社区提出了一系列方法，包括无需重放数据的类增量学习这一高要求追求。在本研究中，我们提出了MIND，一种参数隔离方法，旨在显著提升无重放解决方案的性能，并在多个广泛研究的数据集上达到最先进的结果。我们的方法引入了两个主要贡献：两种替代蒸馏程序，显著提高了MIND的效率，增加了每个子网络的累积知识；以及子网络内跨任务批归一化层的优化。总体而言，MIND在无重放类增量学习中超越了所有最先进的方法（在CIFAR-100/10上分类准确率提升约+6%，在TinyImageNet/10上提升约+10%），在域增量场景中达到约+40%的准确率。此外，我们对每个贡献进行了消融实验，以证明其对性能提升的影响。我们的结果展示了MIND的优越性能，表明其在资源受限环境中应对类增量和域增量学习挑战的潜力。
英文摘要：
The recent surge of pervasive devices that generate dynamic data streams has underscored the necessity for learning systems to adapt continually to data distributional shifts. To tackle this challenge, the research community has put forth a spectrum of methodologies, including the demanding pursuit of class-incremental learning without replay data. In this study, we present MIND, a parameter isolation method that aims to significantly enhance the performance of replay-free solutions and achieve state-of-the-art results on several widely studied datasets. Our approach introduces two main contributions: two alternative distillation procedures that significantly improve the efficiency of MIND increasing the accumulated knowledge of each sub-network, and the optimization of the BachNorm layers across tasks inside the sub-networks. Overall, MIND outperforms all the state-of-the-art methods for rehearsal-free Class-Incremental learning (with an increment in classification accuracy of approx. +6% on CIFAR-100/10 and +10% on TinyImageNet/10) reaching up to approx. +40% accuracy in Domain-Incremental scenarios. Moreover, we ablated each contribution to demonstrate its impact on performance improvement. Our results showcase the superior performance of MIND indicating its potential for addressing the challenges posed by Class-incremental and Domain-Incremental learning in resource-constrained environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
