Harnessing Neural Unit Dynamics for Effective and Scalable Class-Incremental Learning
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；School of Information and Control Engineering, China University of Mining and Technology（中国矿业大学信息与控制工程学院）
Class-incremental learning (CIL) aims to train a model to learn new classes from non-stationary data streams without forgetting old ones. In this paper, we propose a new kind of connectionist model by tailoring neural unit dynamics that adapt the behavior of neural networks for CIL. In each training session, it introduces a supervisory mechanism to guide network expansion whose growth size is compactly commensurate with the intrinsic complexity of a newly arriving task. This constructs a near-minimal network while allowing the model to expand its capacity when cannot sufficiently hold new classes. At inference time, it automatically reactivates the required neural units to retrieve knowledge and leaves the remaining inactivated to prevent interference. We name our model AutoActivator, which is effective and scalable. To gain insights into the neural unit dynamics, we theoretically analyze the model's convergence property via a universal approximation theorem on learning sequential mappings, which is under-explored in the CIL community. Experiments show that our method achieves strong CIL performance in rehearsal-free and minimal-expansion settings with different backbones.
SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning
City University of Hong Kong（香港城市大学）；Harvard University（哈佛大学）；Xi’an Jiaotong University（西安交通大学）；Tencent AI Lab（腾讯AI实验室）；Zhejiang University（浙江大学）；Pengcheng Laboratory（鹏城实验室）
2-speed network ensemble for efficient classification of incremental land-use/land-cover satellite image chips
University of Technology Sydney（悉尼科技大学）；IBM Australia Limited（IBM澳大利亚有限公司）；University of New England（新英格兰大学）；King Abdulaziz University（阿卜杜勒阿齐兹国王大学）；Universiti Kebangsaan Malaysia（马来西亚国民大学）
不断增长的卫星图像数据量对行业和政府提出了挑战，这些行业和政府在及时分析超大数据集的基础上做出数据驱动的决策。用于卫星图像自动分类的常用深度学习算法是时间和资源密集型的训练。当新的图像数据和/或类被添加到训练语料库时，大数据背景下的再训练成本提出了实际挑战。认识到需要一个适应性强，准确，可扩展的卫星图像芯片分类方案，在这项研究中，我们提出了一个合奏：i）缓慢的训练，但高精度的视觉Transformer ;和ii）快速训练，低参数卷积神经网络。视觉Transformer模型提供了可扩展且准确的基础模型。高速CNN提供了一种将新标记的数据纳入分析的有效方法，但以较低的准确性为代价。为了模拟增量数据，将非常大的（约400，000张图像）So2 Sat LCZ 42卫星图像芯片数据集分为四个间隔，每个间隔重新训练高速CNN，每半个间隔训练视觉Transformer。这种实验设置模拟了数据量和多样性随时间的增加。对于自动土地覆盖/土地利用分类的任务，每个数据增量的集成模型优于每个组件模型，最好的准确度为65%，对一个保持测试分区的So2 Sat数据集。建议的合奏和交错的训练时间表提供了一个可扩展的和具有成本效益的卫星图像分类计划，优化处理非常大量的卫星数据。
Generative Feature Replay For Class-Incremental Learning
Computer Vision Center, Universitat Autonoma de Barcelona（巴塞罗那自治大学计算机视觉中心）；Media Integration and Communication Center, University of Florence（佛罗伦萨大学媒体集成与传播中心）；Huawei Kirin Solution（华为麒麟解决方案）
