探索视觉Transformer中的Kolmogorov-Arnold网络扩展以缓解持续学习中的灾难性遗忘
Exploring Kolmogorov-Arnold Network Expansions in Vision Transformers for Mitigating Catastrophic Forgetting in Continual Learning
AI总结：
本文提出用Kolmogorov-Arnold网络替换视觉Transformer中的多层感知机，通过样条激活的局部可塑性缓解持续学习中的灾难性遗忘，并在MNIST和CIFAR100上验证其优于传统MLP方法。
摘要：
持续学习（CL）是指模型在不遗忘先前已获得知识的情况下学习新任务的能力，这仍然是人工智能中的一个关键挑战，尤其是对于利用多层感知机（MLPs）进行全局表示学习的视觉Transformer（ViTs）而言。灾难性遗忘，即新信息覆盖先前知识，在这些模型中尤为严重。本研究提出用Kolmogorov-Arnold网络（KANs）替换ViTs中的MLPs以解决该问题。KANs通过基于样条的激活函数利用局部可塑性，确保每个样本只更新参数的一个子集，从而保留先前学到的知识。本研究在基准数据集（MNIST、CIFAR100）上考察了基于KAN的ViTs在持续学习场景中的有效性，重点关注其在适应新任务的同时保持早期任务准确率的能力。实验结果表明，基于KAN的ViTs显著缓解了灾难性遗忘，在知识保留和任务适应方面优于传统的基于MLP的ViTs。将KANs创新性地集成到ViTs中，为面向动态环境的更鲁棒、更具适应性的模型迈出了有前景的一步。
英文摘要：
Continual learning (CL), the ability of a model to learn new tasks without forgetting previously acquired knowledge, remains a critical challenge in artificial intelligence, particularly for vision transformers (ViTs) utilizing Multilayer Perceptrons (MLPs) for global representation learning. Catastrophic forgetting, where new information overwrites prior knowledge, is especially problematic in these models. This research proposes replacing MLPs in ViTs with Kolmogorov-Arnold Network (KANs) to address this issue. KANs leverage local plasticity through spline-based activations, ensuring that only a subset of parameters is updated per sample, thereby preserving previously learned knowledge. The study investigates the efficacy of KAN-based ViTs in CL scenarios across benchmark datasets (MNIST, CIFAR100), focusing on their ability to retain accuracy on earlier tasks while adapting to new ones. Experimental results demonstrate that KAN-based ViTs significantly mitigate catastrophic forgetting, outperforming traditional MLP-based ViTs in knowledge retention and task adaptation. This novel integration of KANs into ViTs represents a promising step toward more robust and adaptable models for dynamic environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
