Continual HyperTransformer：一种用于持续小样本学习的元学习器
Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learning
AI总结：
针对持续小样本学习的不遗忘问题，提出基于HyperTransformer的递归权重复用方法，无需重放缓冲区等常用机制，可在多种学习场景下保留过往任务知识。
摘要：
我们聚焦于从依次到达的多个任务中实现学习且不遗忘的问题，其中每个任务由包含新类别或已见过类别的小样本情节定义。我们采用最新发表的HyperTransformer（HT，一种基于Transformer的超网络，可直接从支持集生成专用的任务特定CNN权重）来解决该问题。为了从持续的任务序列中学习，我们提出将生成的权重递归复用为下一个任务的HT输入。通过这种方式，生成的CNN权重本身可作为已学习任务的表征，且HT经过训练可更新这些权重，从而在学习新任务时不遗忘过往任务。该方法不同于大多数通常依赖重放缓冲区、权重正则化或与任务相关的架构变更的持续学习算法。我们证明，所提出的配备原型损失的Continual HyperTransformer方法能够在多种场景下学习并保留过往任务的知识，包括从小批次学习、任务增量学习和类别增量学习场景。
英文摘要：
We focus on the problem of learning without forgetting from multiple tasks arriving sequentially, where each task is defined using a few-shot episode of novel or already seen classes. We approach this problem using the recently published HyperTransformer (HT), a Transformer-based hypernetwork that generates specialized task-specific CNN weights directly from the support set. In order to learn from a continual sequence of tasks, we propose to recursively re-use the generated weights as input to the HT for the next task. This way, the generated CNN weights themselves act as a representation of previously learned tasks, and the HT is trained to update these weights so that the new task can be learned without forgetting past tasks. This approach is different from most continual learning algorithms that typically rely on using replay buffers, weight regularization or task-dependent architectural changes. We demonstrate that our proposed Continual HyperTransformer method equipped with a prototypical loss is capable of learning and retaining knowledge about past tasks for a variety of scenarios, including learning from mini-batches, and task-incremental and class-incremental learning scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
