基于Dirichlet生成的持续学习
Continual Learning with Dirichlet Generative-based Rehearsal
AI总结：
提出Dirichlet持续学习（DCL）方法，利用Dirichlet分布建模CVAE潜变量并引入JSKD知识蒸馏，有效解决面向任务对话系统持续学习中的灾难性遗忘问题。
摘要：
数据驱动的面向任务的对话系统（ToDs）的最新进展在应对增量学习时，受限于计算约束和耗时问题而面临困难。持续学习（CL）试图通过避免密集的预训练来解决这一问题，但它面临着灾难性遗忘（CF）的问题。尽管基于生成的排练CL方法已取得显著进展，但生成能准确反映底层任务特定分布的伪样本仍是一个挑战。在本文中，我们提出了Dirichlet持续学习（DCL），一种用于CL的新型基于生成的排练策略。不同于条件变分自编码器（CVAE）中传统使用的高斯潜变量，DCL利用Dirichlet分布的灵活性和多功能性来对潜在先验变量进行建模。这使其能够高效捕获先前任务的句子级特征，并有效指导伪样本的生成。此外，我们引入了Jensen-Shannon知识蒸馏（JSKD），这是一种基于logit的鲁棒知识蒸馏方法，可增强伪样本生成过程中的知识转移。我们的实验证实了该方法在意图检测和槽位填充任务中的有效性，其表现优于现有最先进的方法。
英文摘要：
Recent advancements in data-driven task-oriented dialogue systems (ToDs) struggle with incremental learning due to computational constraints and time-consuming issues. Continual Learning (CL) attempts to solve this by avoiding intensive pre-training, but it faces the problem of catastrophic forgetting (CF). While generative-based rehearsal CL methods have made significant strides, generating pseudo samples that accurately reflect the underlying task-specific distribution is still a challenge. In this paper, we present Dirichlet Continual Learning (DCL), a novel generative-based rehearsal strategy for CL. Unlike the traditionally used Gaussian latent variable in the Conditional Variational Autoencoder (CVAE), DCL leverages the flexibility and versatility of the Dirichlet distribution to model the latent prior variable. This enables it to efficiently capture sentence-level features of previous tasks and effectively guide the generation of pseudo samples. In addition, we introduce Jensen-Shannon Knowledge Distillation (JSKD), a robust logit-based knowledge distillation method that enhances knowledge transfer during pseudo sample generation. Our experiments confirm the efficacy of our approach in both intent detection and slot-filling tasks, outperforming state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
