持续对比式口语理解
Continual Contrastive Spoken Language Understanding
AI总结：
本文研究口语理解在类增量学习设置下的持续学习问题，提出结合经验回放与对比学习的COCONUT方法，通过监督对比损失保留旧知识并利用多模态对比损失学习新数据，在SLU数据集上显著超越基线。
摘要：
近年来，神经网络在各个领域展现出令人瞩目的进展，语音处理也不例外。然而，该领域的近期突破需要使用大型数据集和海量计算资源进行广泛的离线训练。不幸的是，这些模型在持续学习新任务时难以保留先前获取的知识，而从头开始重新训练几乎总是不切实际的。在本文中，我们研究了在类增量学习（CIL）设置下学习用于口语理解的序列到序列模型的问题，并提出了 COCONUT，这是一种依赖经验回放和对比学习相结合的 CIL 方法。通过对仅应用于排练样本的标准监督对比损失进行修改，COCONUT 通过拉近同类样本并推开其他样本来保留已学习的表示。此外，我们利用多模态对比损失，通过对齐音频和文本特征，帮助模型学习新数据更具判别性的表示。我们还研究了不同的对比设计，以将对比损失的优势与用于知识蒸馏的师生架构相结合。在两个已建立的 SLU 数据集上的实验表明了我们提出方法的有效性，以及相对于基线的显著改进。我们还展示了 COCONUT 可以与在模型解码器侧操作的方法相结合，从而带来进一步的指标改进。
英文摘要：
Recently, neural networks have shown impressive progress across diverse fields, with speech processing being no exception. However, recent breakthroughs in this area require extensive offline training using large datasets and tremendous computing resources. Unfortunately, these models struggle to retain their previously acquired knowledge when learning new tasks continually, and retraining from scratch is almost always impractical. In this paper, we investigate the problem of learning sequence-to-sequence models for spoken language understanding in a class-incremental learning (CIL) setting and we propose COCONUT, a CIL method that relies on the combination of experience replay and contrastive learning. Through a modified version of the standard supervised contrastive loss applied only to the rehearsal samples, COCONUT preserves the learned representations by pulling closer samples from the same class and pushing away the others. Moreover, we leverage a multimodal contrastive loss that helps the model learn more discriminative representations of the new data by aligning audio and text features. We also investigate different contrastive designs to combine the strengths of the contrastive loss with teacher-student architectures used for distillation. Experiments on two established SLU datasets reveal the effectiveness of our proposed approach and significant improvements over the baselines. We also show that COCONUT can be combined with methods that operate on the decoder side of the model, resulting in further metrics improvements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
