arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.09910cs.LGstat.ML
收藏

使用单梯度步更新进行多任务学习以实现任务平衡

Multitask Learning with Single Gradient Step Update for Task Balancing

Dongguk University - Seoul(东国大学首尔校区)

Sungjae Lee, Youngdoo Son

摘要:

多任务学习是一种提高泛化性能并降低计算强度和内存使用的方法。然而,同时学习多个任务可能比学习单个任务更困难,因为它会导致任务之间的不平衡。为了解决不平衡问题,我们提出了一种算法,通过将基于梯度的元学习应用于多任务学习,在梯度级别上平衡任务之间的关系。所提出的方法分别训练共享层和任务特定层,以便在多任务网络中具有不同角色的两个层可以适合各自的目的。特别地,包含任务之间共享的信息性知识的共享层通过采用单梯度步长更新和内/外循环训练来训练,以减轻梯度级的不平衡问题。我们将所提出的方法应用于各种多任务计算机视觉问题,并实现了最先进的性能。

英文摘要:

Multitask learning is a methodology to boost generalization performance and also reduce computational intensity and memory usage. However, learning multiple tasks simultaneously can be more difficult than learning a single task because it can cause imbalance among tasks. To address the imbalance problem, we propose an algorithm to balance between tasks at the gradient level by applying gradient-based meta-learning to multitask learning. The proposed method trains shared layers and task-specific layers separately so that the two layers with different roles in a multitask network can be fitted to their own purposes. In particular, the shared layer that contains informative knowledge shared among tasks is trained by employing single gradient step update and inner/outer loop training to mitigate the imbalance problem at the gradient level. We apply the proposed method to various multitask computer vision problems and achieve state-of-the-art performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑