批量模型整合：一种多任务模型整合框架
Batch Model Consolidation: A Multi-Task Model Consolidation Framework
AI总结：
本文提出批量模型整合框架BMC，通过并行训练多个专家模型并在内存数据上批量整合知识，在71个多样化图像分类任务上显著优于现有持续学习方法。
摘要：
在持续学习（CL）中，模型需要顺序学习一系列任务，同时不能对先前学到的任务造成显著的性能下降。当前的方法在面对来自不同领域和不同难度的长任务序列时表现不佳。许多现有的持续学习方法由于过高的内存开销或训练时间，或者与单一设备紧密耦合，难以在实际中应用。基于广泛应用的mini-batch训练所带来的直觉，我们提出了批量模型整合（BMC），以支持在多个智能体面对一系列任务的条件下实现更现实的持续学习。在正则化阶段，BMC在一组不相交的任务上并行训练多个专家模型。每个专家通过稳定性损失保持与基础模型的权重相似性，并从任务数据的一小部分构建缓冲区。在整合阶段，我们使用批量整合损失在聚合所有缓冲区的内存数据上，对一批专家模型所学的知识进行合并。我们在消融研究中彻底评估了方法的每个组成部分，并在标准化基准数据集Split-CIFAR-100、Tiny-ImageNet以及由71个来自不同领域和难度的图像分类任务组成的Stream数据集上证明了其有效性。我们的方法比次优的持续学习方法性能高出70%，并且是唯一能够在71个任务结束时仍保持性能的方法；我们的基准可以在https://github.com/fostiropoulos/stream_benchmark访问。
英文摘要：
In Continual Learning (CL), a model is required to learn a stream of tasks sequentially without significant performance degradation on previously learned tasks. Current approaches fail for a long sequence of tasks from diverse domains and difficulties. Many of the existing CL approaches are difficult to apply in practice due to excessive memory cost or training time, or are tightly coupled to a single device. With the intuition derived from the widely applied mini-batch training, we propose Batch Model Consolidation ($\textbf{BMC}$) to support more realistic CL under conditions where multiple agents are exposed to a range of tasks. During a $\textit{regularization}$ phase, BMC trains multiple $\textit{expert models}$ in parallel on a set of disjoint tasks. Each expert maintains weight similarity to a $\textit{base model}$ through a $\textit{stability loss}$, and constructs a $\textit{buffer}$ from a fraction of the task's data. During the $\textit{consolidation}$ phase, we combine the learned knowledge on 'batches' of $\textit{expert models}$ using a $\textit{batched consolidation loss}$ in $\textit{memory}$ data that aggregates all buffers. We thoroughly evaluate each component of our method in an ablation study and demonstrate the effectiveness on standardized benchmark datasets Split-CIFAR-100, Tiny-ImageNet, and the Stream dataset composed of 71 image classification tasks from diverse domains and difficulties. Our method outperforms the next best CL approach by 70% and is the only approach that can maintain performance at the end of 71 tasks; Our benchmark can be accessed at https://github.com/fostiropoulos/stream_benchmark
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
