基于互补记忆系统的开放词汇分类持续学习
Continual Learning in Open-vocabulary Classification with Complementary Memory Systems
AI总结：
受人类互补学习系统启发，提出结合CLIP零样本模型与样本模型预测、辅以树探针方法的开放词汇持续学习方案，在多增量设置下实现学习速度与两类有效性的良好平衡。
摘要：
我们受人类认知中观察到的互补学习系统启发，提出了一种灵活高效的开放词汇图像分类持续学习方法。具体而言，我们提出结合CLIP零样本模型与基于样本的模型的预测，利用零样本估计的样本类别属于样本类别的概率进行融合。我们还提出了“树探针（tree probe）”方法，它是对惰性学习原理的改进，能够从新样本中快速学习，且准确率与批量训练的线性模型相当。我们在数据增量、类别增量和任务增量设置下进行了测试，同时测试了在零样本类别与已学习类别的不同子集上执行灵活推理的能力。我们提出的方法在学习速度、目标任务有效性和零样本有效性之间实现了良好的平衡。代码将发布于https://github.com/jessemelpolio/TreeProbe。
英文摘要：
We introduce a method for flexible and efficient continual learning in open-vocabulary image classification, drawing inspiration from the complementary learning systems observed in human cognition. Specifically, we propose to combine predictions from a CLIP zero-shot model and the exemplar-based model, using the zero-shot estimated probability that a sample's class is within the exemplar classes. We also propose a "tree probe" method, an adaption of lazy learning principles, which enables fast learning from new examples with competitive accuracy to batch-trained linear models. We test in data incremental, class incremental, and task incremental settings, as well as ability to perform flexible inference on varying subsets of zero-shot and learned categories. Our proposed method achieves a good balance of learning speed, target task effectiveness, and zero-shot effectiveness. Code will be available at https://github.com/jessemelpolio/TreeProbe.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
