自己涂鸦:通过画几张草图进行增量学习
Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches
摘要:
人类的视觉系统在仅仅从几个例子中学习新的视觉概念方面是显着的。这正是Few-Shot类增量学习(FSCIL)背后的目标,其中的重点是确保模型不会遭受“遗忘”。在本文中,我们通过解决瓶颈其无处不在的应用的两个关键问题来进一步推动FSCIL的边界(i)模型可以从不同的模式中学习,而不仅仅是照片(就像人类一样),以及(ii)如果照片不容易访问(由于道德和隐私限制)。我们的主要创新在于倡导使用草图作为课堂支持的新模式。该产品是一个“自己涂鸦”(DIY)的FSCIL框架,用户可以自由地勾勒出一个新类的几个例子,让模型学习识别该类的照片。为此,我们提出了一个框架,注入(i)梯度共识域不变学习,(ii)知识蒸馏保留旧类信息,和(iii)图注意力网络之间的旧类和新类的消息传递。我们的实验表明,草图是更好的类支持比文本的背景下FSCIL,呼应其他地方的素描文学的研究结果。
英文摘要:
The human visual system is remarkable in learning new visual concepts from just a few examples. This is precisely the goal behind few-shot class incremental learning (FSCIL), where the emphasis is additionally placed on ensuring the model does not suffer from "forgetting". In this paper, we push the boundary further for FSCIL by addressing two key questions that bottleneck its ubiquitous application (i) can the model learn from diverse modalities other than just photo (as humans do), and (ii) what if photos are not readily accessible (due to ethical and privacy constraints). Our key innovation lies in advocating the use of sketches as a new modality for class support. The product is a "Doodle It Yourself" (DIY) FSCIL framework where the users can freely sketch a few examples of a novel class for the model to learn to recognize photos of that class. For that, we present a framework that infuses (i) gradient consensus for domain invariant learning, (ii) knowledge distillation for preserving old class information, and (iii) graph attention networks for message passing between old and novel classes. We experimentally show that sketches are better class support than text in the context of FSCIL, echoing findings elsewhere in the sketching literature.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。