arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.05495cs.LGcs.AIcs.CL
收藏

LSEBMCL:一种用于持续学习的潜空间基于能量模型

LSEBMCL: A Latent Space Energy-Based Model for Continual Learning

The University of Texas at Dallas(德克萨斯大学达拉斯分校);Vextex AI(Vextex AI公司);Google(谷歌公司);Texas A&M University(德克萨斯农工大学)

Xiaodi Li, Dingcheng Li, Rujun Gao, Mahmoud Zamani, Latifur Khan

AI总结:

本文提出 LSEBMCL,将 EBM 层作为 NLP 持续学习框架的外部生成器,通过从旧任务采样数据缓解灾难性遗忘,并在全部实验中达到 SOTA。

摘要:

持续学习在在线新闻摘要和产品分类等许多实际应用中已变得不可或缺。其主要挑战被称为灾难性遗忘,即模型在新任务上训练时会无意中丢弃先前学到的知识的现象。现有解决方案包括存储来自先前类别的样本、在微调过程中对参数进行正则化,或为每个任务分配不同的模型参数。本工作提出的解决方案 LSEBMCL(用于持续学习的潜空间基于能量模型)是利用基于能量模型(EBM),通过在训练新任务时从先前任务采样数据点来防止灾难性遗忘。EBM 是一种为每个输入数据点关联一个能量值的机器学习模型。所提方法在面向 NLP 任务的持续学习框架中使用一个 EBM 层作为外部生成器。该研究证明了 EBM 在 NLP 任务中的有效性,并在所有实验中取得了最先进的结果。

英文摘要:

Continual learning has become essential in many practical applications such as online news summaries and product classification. The primary challenge is known as catastrophic forgetting, a phenomenon where a model inadvertently discards previously learned knowledge when it is trained on new tasks. Existing solutions involve storing exemplars from previous classes, regularizing parameters during the fine-tuning process, or assigning different model parameters to each task. The proposed solution LSEBMCL (Latent Space Energy-Based Model for Continual Learning) in this work is to use energy-based models (EBMs) to prevent catastrophic forgetting by sampling data points from previous tasks when training on new ones. The EBM is a machine learning model that associates an energy value with each input data point. The proposed method uses an EBM layer as an outer-generator in the continual learning framework for NLP tasks. The study demonstrates the efficacy of EBM in NLP tasks, achieving state-of-the-art results in all experiments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑