arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.01853cs.LGcs.AI
收藏

具有分布偏移的可解释持续学习基的混合

Mixture of basis for interpretable continual learning with distribution shifts

JPMorgan AI Research(摩根大通人工智能研究院)

Mengda Xu, Sumitra Ganesh, Pranay Pasula

摘要:

在数据分布不断变化的环境中进行持续学习是一个具有挑战性的问题。在本文中,我们考虑的设置中,数据分布(任务)的转变突然和这些转变的时间是未知的。此外,我们考虑了一个半监督的任务不可知的设置,其中学习算法可以访问任务分段和未分段的数据进行离线训练。我们提出了一种新的方法称为Basismodels(MoB)的混合物,用于解决这个问题的设置。其核心思想是学习一小部分基本模型,并构建一个动态的、依赖于任务的模型混合体来预测当前任务。我们还提出了一种新的方法来检测的观察是出分布相对于现有的基础模型,并根据需要实例化新的模型。我们在多个领域测试了我们的方法,并表明它在大多数情况下比现有方法获得更好的预测误差,同时使用比其他多模型方法更少的模型。此外,我们分析了潜在的任务表征的MoB学习,并表明,类似的任务往往集群在潜在的空间和任务边界时,任务是不相似的潜在表示转移。

英文摘要:

Continual learning in environments with shifting data distributions is a challenging problem with several real-world applications. In this paper we consider settings in which the data distribution(task) shifts abruptly and the timing of these shifts are not known. Furthermore, we consider a semi-supervised task-agnostic setting in which the learning algorithm has access to both task-segmented and unsegmented data for offline training. We propose a novel approach called mixture of Basismodels (MoB) for addressing this problem setting. The core idea is to learn a small set of basis models and to construct a dynamic, task-dependent mixture of the models to predict for the current task. We also propose a new methodology to detect observations that are out-of-distribution with respect to the existing basis models and to instantiate new models as needed. We test our approach in multiple domains and show that it attains better prediction error than existing methods in most cases while using fewer models than other multiple model approaches. Moreover, we analyze the latent task representations learned by MoB and show that similar tasks tend to cluster in the latent space and that the latent representation shifts at the task boundaries when tasks are dissimilar.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑