RotoGrad：多任务学习中的梯度均匀化
RotoGrad: Gradient Homogenization in Multitask Learning
摘要：
多任务学习越来越多地被应用于计算机视觉和强化学习等应用领域。然而，由于负迁移的影响，最佳地利用其优势仍然是一个重大挑战。以前的工作已经将这个问题归结为优化共享网络参数时跨任务的梯度幅度和方向的差异。虽然最近的工作已经承认，负迁移是一个双重的问题，现有的方法不足，因为他们只专注于要么跨越任务的梯度大小;或greenly改变梯度方向，忽视未来的冲突。在这项工作中，我们介绍了RotoGrad，一种整体解决负迁移的算法：它联合均匀梯度幅度和方向，同时确保训练收敛。我们证明了RotoGrad在复杂问题中优于竞争方法，包括CelebA中的多标签分类和NYUv 2数据集中的计算机视觉任务。Pytorch实现可以在https://github.com/adrianjav/rotograd中找到。
英文摘要：
Multitask learning is being increasingly adopted in applications domains like computer vision and reinforcement learning. However, optimally exploiting its advantages remains a major challenge due to the effect of negative transfer. Previous works have tracked down this issue to the disparities in gradient magnitudes and directions across tasks, when optimizing the shared network parameters. While recent work has acknowledged that negative transfer is a two-fold problem, existing approaches fall short as they only focus on either homogenizing the gradient magnitude across tasks; or greedily change the gradient directions, overlooking future conflicts. In this work, we introduce RotoGrad, an algorithm that tackles negative transfer as a whole: it jointly homogenizes gradient magnitudes and directions, while ensuring training convergence. We show that RotoGrad outperforms competing methods in complex problems, including multi-label classification in CelebA and computer vision tasks in the NYUv2 dataset. A Pytorch implementation can be found in https://github.com/adrianjav/rotograd.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
