多任务学习中任务难度平衡的一种简单通用方法
A Simple General Approach to Balance Task Difficulty in Multi-Task Learning
摘要：
在多任务学习中，不同任务的难度是不同的。有许多工作来处理这种情况，我们把它们分为五类，包括直接和方法，加权和方法，最大值方法，课程学习方法，和多目标优化方法。这些方法有其自身的局限性，例如，使用手动设计的规则来更新任务权重，非平滑的目标函数，以及未能包含除训练损失之外的其他函数。在本文中，为了减轻这些限制，我们提出了一个平衡的多任务学习（BMTL）框架。与现有的依赖于任务权重的研究不同，BMTL框架提出将每个任务的训练损失转换为任务之间的难度水平，基于一种直观的想法，即训练损失较大的任务在优化过程中会受到更多的关注。我们分析了转换函数，并推导出必要条件。所提出的BMTL框架非常简单，可以与大多数多任务学习模型相结合。实证研究表明，国家的最先进的性能所提出的BMTL框架。
英文摘要：
In multi-task learning, difficulty levels of different tasks are varying. There are many works to handle this situation and we classify them into five categories, including the direct sum approach, the weighted sum approach, the maximum approach, the curriculum learning approach, and the multi-objective optimization approach. Those approaches have their own limitations, for example, using manually designed rules to update task weights, non-smooth objective function, and failing to incorporate other functions than training losses. In this paper, to alleviate those limitations, we propose a Balanced Multi-Task Learning (BMTL) framework. Different from existing studies which rely on task weighting, the BMTL framework proposes to transform the training loss of each task to balance difficulty levels among tasks based on an intuitive idea that tasks with larger training losses will receive more attention during the optimization procedure. We analyze the transformation function and derive necessary conditions. The proposed BMTL framework is very simple and it can be combined with most multi-task learning models. Empirical studies show the state-of-the-art performance of the proposed BMTL framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
