arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2506.02065cs.LGcs.AIcs.CV
收藏

EWGN:深度学习中的弹性权重生成与上下文切换

EWGN: Elastic Weight Generation and Context Switching in Deep Learning

Department of Computer Science and Engineering(计算机科学与工程系);Indian Institute of Technology Gandhinagar(印度理工学院冈丁纳加);Department of Cognitive Science(认知科学系)

Shriraj P. Sawant, Krishna P. Miyapuram

AI总结:

本文提出弹性权重生成网络EWGN,通过额外网络根据输入动态生成主网络权重,实现任务间上下文切换,以缓解灾难性遗忘并提升持续学习性能。

摘要:

学习并保留多种多样任务的能力是人类智能的一个标志,这一能力启发着通用人工智能的研究。持续学习方法为实现这一目标迈出了重要一步。众所周知,任务变异性和上下文切换对神经网络学习具有挑战性。灾难性遗忘是指在网络学习新任务时,对先前已学任务的保持性能变差。在不同任务上下文之间进行切换,可以通过防止网络不同任务权重之间的相互干扰,成为缓解该问题的一种有效方法。本文提出弹性权重生成网络(EWGN),作为一种在两个不同任务之间进行上下文切换的思路。所提出的EWGN架构使用一个额外的网络,在巩固已学权重的同时动态生成主网络的权重。权重生成依赖于输入,因此能够实现上下文切换。我们使用标准计算机视觉数据集MNIST和fashion-MNIST,分析了在随机梯度下降和弹性权重巩固学习算法下,全连接网络、卷积神经网络和EWGN架构对先前已学任务表示的保持能力。理解动态权重生成和上下文切换能力,有助于实现持续学习,从而提升性能。

英文摘要:

The ability to learn and retain a wide variety of tasks is a hallmark of human intelligence that has inspired research in artificial general intelligence. Continual learning approaches provide a significant step towards achieving this goal. It has been known that task variability and context switching are challenging for learning in neural networks. Catastrophic forgetting refers to the poor performance on retention of a previously learned task when a new task is being learned. Switching between different task contexts can be a useful approach to mitigate the same by preventing the interference between the varying task weights of the network. This paper introduces Elastic Weight Generative Networks (EWGN) as an idea for context switching between two different tasks. The proposed EWGN architecture uses an additional network that generates the weights of the primary network dynamically while consolidating the weights learned. The weight generation is input-dependent and thus enables context switching. Using standard computer vision datasets, namely MNIST and fashion-MNIST, we analyse the retention of previously learned task representations in Fully Connected Networks, Convolutional Neural Networks, and EWGN architectures with Stochastic Gradient Descent and Elastic Weight Consolidation learning algorithms. Understanding dynamic weight generation and context-switching ability can be useful in enabling continual learning for improved performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑