arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1609.09563cs.LGcs.DC
收藏

异步多任务学习

Asynchronous Multi-Task Learning

Michigan State University(密歇根州立大学)

Inci M. Baytas, Ming Yan, Anil K. Jain, Jiayu Zhou

摘要:

许多现实世界的机器学习应用程序涉及多个相互关联的学习任务。例如,在医疗保健领域,我们需要为许多医院学习某种疾病的预测模型。由于患者人群分布的固有差异,每家医院的模型可能不同。然而,由于对同一疾病建模的学习任务的性质,这些模型也密切相关。多任务学习(MTL)范式通过同时学习所有任务,在任务之间进行归纳知识转移,以提高泛化性能。当用于学习任务的数据集存储在不同位置时,由于诸如高数据量和隐私的各种实际问题,传输数据以提供数据集中式计算环境可能并不总是可行的。在本文中,我们提出了一个原则性的MTL框架的分布式和异步优化,以解决上述挑战。在我们的框架中,梯度更新不等待收集所有任务的梯度信息。因此,所提出的方法是非常有效的,当通信延迟过高的一些任务节点。我们表明,许多正则化MTL配方可以受益于这个框架,包括低秩MTL共享子空间学习。对合成数据集和真实数据集的实证研究表明了所提出的框架的效率和有效性。

英文摘要:

Many real-world machine learning applications involve several learning tasks which are inter-related. For example, in healthcare domain, we need to learn a predictive model of a certain disease for many hospitals. The models for each hospital may be different because of the inherent differences in the distributions of the patient populations. However, the models are also closely related because of the nature of the learning tasks modeling the same disease. By simultaneously learning all the tasks, multi-task learning (MTL) paradigm performs inductive knowledge transfer among tasks to improve the generalization performance. When datasets for the learning tasks are stored at different locations, it may not always be feasible to transfer the data to provide a data-centralized computing environment due to various practical issues such as high data volume and privacy. In this paper, we propose a principled MTL framework for distributed and asynchronous optimization to address the aforementioned challenges. In our framework, gradient update does not wait for collecting the gradient information from all the tasks. Therefore, the proposed method is very efficient when the communication delay is too high for some task nodes. We show that many regularized MTL formulations can benefit from this framework, including the low-rank MTL for shared subspace learning. Empirical studies on both synthetic and real-world datasets demonstrate the efficiency and effectiveness of the proposed framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑