预训练特征提取器中的适配器学习用于疾病持续学习
Adapter Learning in Pretrained Feature Extractor for Continual Learning of Diseases
AI总结：
针对智能诊断系统持续学习新疾病时的灾难性遗忘问题，提出ACL框架，通过在固定预训练特征提取器上添加轻量任务特定适配器并协同微调任务头，实现新疾病的有效持续学习。
摘要：
目前，智能诊断系统在部署后缺乏持续学习诊断新疾病的能力，同时还要保留旧疾病知识。特别是，用新疾病的训练数据更新智能诊断系统会导致对旧疾病知识的灾难性遗忘。为了解决灾难性遗忘问题，提出了一种基于适配器的持续学习框架，称为ACL，以帮助在每一轮（或任务）持续学习中有效地学习一组新疾病，而不改变共享特征提取器。可学习的轻量级任务特定适配器可以灵活设计（例如，两个卷积层），然后添加到预训练且固定的特征提取器上。与一个专门设计的任务特定头一起，该头将所有先前学习过的旧疾病吸收为一个单一的“分布外”类别，任务特定适配器可以帮助预训练特征提取器更有效地提取疾病之间的判别特征。此外，采用一种简单而有效的微调方法，协同微调多个任务特定头，使得不同头的输出具有可比性，从而在模型推理过程中能够更准确地选择合适的分类器头。在三个图像数据集上的大量实证评估证明了ACL在新疾病持续学习中的优越性能。源代码可在https://github.com/GiantJun/CL_Pytorch获取。
英文摘要：
Currently intelligent diagnosis systems lack the ability of continually learning to diagnose new diseases once deployed, under the condition of preserving old disease knowledge. In particular, updating an intelligent diagnosis system with training data of new diseases would cause catastrophic forgetting of old disease knowledge. To address the catastrophic forgetting issue, an Adapter-based Continual Learning framework called ACL is proposed to help effectively learn a set of new diseases at each round (or task) of continual learning, without changing the shared feature extractor. The learnable lightweight task-specific adapter(s) can be flexibly designed (e.g., two convolutional layers) and then added to the pretrained and fixed feature extractor. Together with a specially designed task-specific head which absorbs all previously learned old diseases as a single "out-of-distribution" category, task-specific adapter(s) can help the pretrained feature extractor more effectively extract discriminative features between diseases. In addition, a simple yet effective fine-tuning is applied to collaboratively fine-tune multiple task-specific heads such that outputs from different heads are comparable and consequently the appropriate classifier head can be more accurately selected during model inference. Extensive empirical evaluations on three image datasets demonstrate the superior performance of ACL in continual learning of new diseases. The source code is available at https://github.com/GiantJun/CL_Pytorch.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
