arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.05820cs.LG
收藏

多任务神经过程

Multi-Task Neural Processes

University of Amsterdam(阿姆斯特丹大学);Inception Institute of Artificial Intelligence(起源人工智能研究院)

Jiayi Shen, Xiantong Zhen, Marcel Worring, Ling Shao

摘要:

神经过程是近年来出现的一类结合了神经网络和随机过程优点的强有力的神经隐变量模型。由于它们可以在网络的函数空间中编码上下文数据,因此它们提供了一种新的方法来建模多任务学习中的任务相关性。为了研究其潜力,我们开发了多任务神经过程,这是多任务学习神经过程的一种新变体。特别是,我们建议在函数空间中探索相关任务中的可转移知识,以提供用于改进每个任务的归纳偏差。要做到这一点,我们推导出的功能先验的分层贝叶斯推理框架,使每个任务,将相关任务提供的共享知识到其上下文的预测功能。我们的多任务神经过程在方法上扩展了香草神经过程的范围,并为多任务学习提供了一种新的方法来探索函数空间中的任务相关性。所提出的多任务神经过程能够在有限的标记数据和存在域转移的情况下学习多任务。我们对多任务回归和分类任务的几个基准进行了广泛的实验评估。结果表明,多任务神经过程在多任务学习任务之间传递有用知识的有效性,以及在多任务分类和脑图像分割方面的优异性能。

英文摘要:

Neural processes have recently emerged as a class of powerful neural latent variable models that combine the strengths of neural networks and stochastic processes. As they can encode contextual data in the network's function space, they offer a new way to model task relatedness in multi-task learning. To study its potential, we develop multi-task neural processes, a new variant of neural processes for multi-task learning. In particular, we propose to explore transferable knowledge from related tasks in the function space to provide inductive bias for improving each individual task. To do so, we derive the function priors in a hierarchical Bayesian inference framework, which enables each task to incorporate the shared knowledge provided by related tasks into its context of the prediction function. Our multi-task neural processes methodologically expand the scope of vanilla neural processes and provide a new way of exploring task relatedness in function spaces for multi-task learning. The proposed multi-task neural processes are capable of learning multiple tasks with limited labeled data and in the presence of domain shift. We perform extensive experimental evaluations on several benchmarks for the multi-task regression and classification tasks. The results demonstrate the effectiveness of multi-task neural processes in transferring useful knowledge among tasks for multi-task learning and superior performance in multi-task classification and brain image segmentation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑