大模型驱动的小样本持续学习
Big-model Driven Few-shot Continual Learning
AI总结：
提出大模型驱动的小样本持续学习框架，利用大模型迁移与自适应蒸馏，在三个数据集上超越现有方法。
摘要：
小样本持续学习（FSCL）近年来吸引了广泛关注并取得了一些进展，但由于仅有少量增量样本的限制，目前很难在准确性上再次取得重大突破。受人类在生活中学习时所展现的独特认知能力的启发，本文提出了一种新颖的大模型驱动的小样本持续学习（B-FSCL）框架，旨在世界大模型（如同人类累积的知识）的牵引下逐步演化模型。具体而言，我们执行大模型驱动的迁移学习，以利用这些现有大模型的强大编码能力，使持续模型能够适应少量新添加的样本，同时避免过拟合问题。考虑到大模型与持续模型对相同图像可能产生不同的感知结果，我们引入了一种实例级自适应决策机制，以提供针对不同样本调整的高灵活性认知支持。反过来，自适应决策可进一步用于优化持续模型的参数，执行大模型知识信息的自适应蒸馏。我们在三个流行数据集（包括CIFAR100、minilmageNet和CUB200）上对所提出的B-FSCL进行了实验，结果完全超越了所有最先进的FSCL方法。
英文摘要：
Few-shot continual learning (FSCL) has attracted intensive attention and achieved some advances in recent years, but now it is difficult to again make a big stride in accuracy due to the limitation of only few-shot incremental samples. Inspired by distinctive human cognition ability in life learning, in this work, we propose a novel Big-model driven Few-shot Continual Learning (B-FSCL) framework to gradually evolve the model under the traction of the world's big-models (like human accumulative knowledge). Specifically, we perform the big-model driven transfer learning to leverage the powerful encoding capability of these existing big-models, which can adapt the continual model to a few of newly added samples while avoiding the over-fitting problem. Considering that the big-model and the continual model may have different perceived results for the identical images, we introduce an instance-level adaptive decision mechanism to provide the high-level flexibility cognitive support adjusted to varying samples. In turn, the adaptive decision can be further adopted to optimize the parameters of the continual model, performing the adaptive distillation of big-model's knowledge information. Experimental results of our proposed B-FSCL on three popular datasets (including CIFAR100, minilmageNet and CUB200) completely surpass all state-of-the-art FSCL methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
