arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1903.09109cs.LGstat.ML
收藏

多任务学习中学习任务相似性的原则性方法

A Principled Approach for Learning Task Similarity in Multitask Learning

Université Laval(拉瓦尔大学);University of Pennsylvania(宾夕法尼亚大学)

Changjian Shui, Mahdieh Abbasi, Louis-Émile Robitaille, Boyu Wang, Christian Gagné

摘要:

多任务学习的目的是同时解决一组相关的任务,通过利用共享的知识来提高单个任务的性能。因此,多任务学习的一个重要方面是理解一组任务中的相似性。先前的工作已经明确地结合了这种相似性信息(例如,每个任务的加权损失)或隐式地(例如,特征适应的对抗性损失),以实现良好的经验性能。然而,添加任务相似性知识的理论动机往往缺失或不完整。本文从理论的角度给出了不同的视角来理解这种做法。我们首先提供了一个上限的泛化误差的多任务学习,显示的好处,明确和隐含的任务相似性知识。我们系统地推导出两个不同的任务相似性度量的基础上的界限:H分歧和Wasserstein距离。基于这些理论结果,我们重新审视了对抗性多任务神经网络,提出了一种新的训练算法来迭代学习任务关联系数和神经网络参数。我们评估我们的新算法经验上的几个基准,不仅表明我们发现有趣的和强大的任务关系,但所提出的方法优于基线,重申算法设计的理论见解的好处。

英文摘要:

Multitask learning aims at solving a set of related tasks simultaneously, by exploiting the shared knowledge for improving the performance on individual tasks. Hence, an important aspect of multitask learning is to understand the similarities within a set of tasks. Previous works have incorporated this similarity information explicitly (e.g., weighted loss for each task) or implicitly (e.g., adversarial loss for feature adaptation), for achieving good empirical performances. However, the theoretical motivations for adding task similarity knowledge are often missing or incomplete. In this paper, we give a different perspective from a theoretical point of view to understand this practice. We first provide an upper bound on the generalization error of multitask learning, showing the benefit of explicit and implicit task similarity knowledge. We systematically derive the bounds based on two distinct task similarity metrics: H divergence and Wasserstein distance. From these theoretical results, we revisit the Adversarial Multi-task Neural Network, proposing a new training algorithm to learn the task relation coefficients and neural network parameters iteratively. We assess our new algorithm empirically on several benchmarks, showing not only that we find interesting and robust task relations, but that the proposed approach outperforms the baselines, reaffirming the benefits of theoretical insight in algorithm design.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑