arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1807.09536cs.CV
收藏

端到端增量学习

End-to-End Incremental Learning

University of Málaga(马拉加大学);University of Córdoba(科尔多瓦大学);Univ. Grenoble Alpes(格勒诺布尔阿尔卑斯大学);Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);Grenoble INP(格勒诺布尔国立高等物理、数学与信息学院)

Francisco M. Castro, Manuel J. Marín-Jiménez, Nicolás Guil, Cordelia Schmid, Karteek Alahari

摘要:

尽管近年来深度学习方法由于其最先进的结果而脱颖而出,但它们仍然遭受灾难性遗忘,当使用增量添加的新类进行训练时,整体性能急剧下降。这是由于当前的神经网络架构需要整个数据集(包括来自旧类和新类的所有样本)来更新模型-随着类的数量增加,这一要求很容易变得不可持续。我们通过增量学习深度神经网络的方法来解决这个问题,使用新数据和仅对应于旧类样本的小样本集。这是基于由保留从旧类获得的知识的蒸馏措施和学习新类的交叉熵损失组成的损失。我们的增量训练是在保持整个框架端到端的同时实现的,即,联合学习数据表示和分类器,不像最近的方法没有这样的保证。我们在CIFAR-100和ImageNet(ILSVRC 2012)图像分类数据集上对我们的方法进行了广泛的评估,并展示了最先进的性能。

英文摘要:

Although deep learning approaches have stood out in recent years due to their state-of-the-art results, they continue to suffer from catastrophic forgetting, a dramatic decrease in overall performance when training with new classes added incrementally. This is due to current neural network architectures requiring the entire dataset, consisting of all the samples from the old as well as the new classes, to update the model -a requirement that becomes easily unsustainable as the number of classes grows. We address this issue with our approach to learn deep neural networks incrementally, using new data and only a small exemplar set corresponding to samples from the old classes. This is based on a loss composed of a distillation measure to retain the knowledge acquired from the old classes, and a cross-entropy loss to learn the new classes. Our incremental training is achieved while keeping the entire framework end-to-end, i.e., learning the data representation and the classifier jointly, unlike recent methods with no such guarantees. We evaluate our method extensively on the CIFAR-100 and ImageNet (ILSVRC 2012) image classification datasets, and show state-of-the-art performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑