CoLoR-GAN：生成对抗网络中基于低秩适应的持续少样本学习
CoLoR-GAN: Continual Few-Shot Learning with Low-Rank Adaptation in Generative Adversarial Networks
AI总结：
提出CoLoR-GAN框架，结合低秩适应与LoRA in LoRA技术，在生成对抗网络中高效实现持续少样本学习，以更少参数达到先进性能。
摘要：
在生成对抗网络（GAN）的背景下，持续学习（CL）仍然是一个具有挑战性的问题，尤其是在从少样本（FS）样本中学习而不发生灾难性遗忘时。当前最有效的先进方法（如LFS-GAN）在每次训练迭代中都会引入不可忽略的新权重数量，从长远来看，这将会变得显著。为此，本文介绍了名为CoLoR-GAN的持续少样本学习与低秩适应GAN框架，该框架旨在同时处理FS和CL，利用低秩张量高效地将模型适应目标任务，同时进一步减少所需参数的数量。应用vanilla LoRA实现已经使我们获得了相当好的结果。为了进一步优化适配器的大小，我们挑战了LoRA的极限，引入了用于卷积层的LoRA in LoRA（LLoRA）技术。最后，考虑到LoRA超参数选择的关键性，我们提供了一项实证研究，以便轻松找到最佳超参数。我们通过在多个基准CL和FS任务上的实验证明了CoLoR-GAN的有效性，并表明我们的模型是高效的，达到了SOTA性能，但所需资源大幅减少。源代码可在Github上获取：https://github.com/munsifali11/CoLoR-GAN。
英文摘要：
Continual learning (CL) in the context of Generative Adversarial Networks (GANs) remains a challenging problem, particularly when it comes to learn from a few-shot (FS) samples without catastrophic forgetting. Current most effective state-of-the-art (SOTA) methods, like LFS-GAN, introduce a non-negligible quantity of new weights at each training iteration, which would become significant when considering the long term. For this reason, this paper introduces \textcolor{red}{\textbf{\underline{c}}}ontinual few-sh\textcolor{red}{\textbf{\underline{o}}}t learning with \textcolor{red}{\textbf{\underline{lo}}}w-\textcolor{red}{\textbf{\underline{r}}}ank adaptation in GANs named CoLoR-GAN, a framework designed to handle both FS and CL together, leveraging low-rank tensors to efficiently adapt the model to target tasks while reducing even more the number of parameters required. Applying a vanilla LoRA implementation already permitted us to obtain pretty good results. In order to optimize even further the size of the adapters, we challenged LoRA limits introducing a LoRA in LoRA (LLoRA) technique for convolutional layers. Finally, aware of the criticality linked to the choice of the hyperparameters of LoRA, we provide an empirical study to easily find the best ones. We demonstrate the effectiveness of CoLoR-GAN through experiments on several benchmark CL and FS tasks and show that our model is efficient, reaching SOTA performance but with a number of resources enormously reduced. Source code is available on \href{https://github.com/munsifali11/CoLoR-GAN}{Github.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
