arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2001.06782cs.LGcs.CVcs.ROstat.ML
收藏

多任务学习的梯度手术

Gradient Surgery for Multi-Task Learning

Stanford University(斯坦福大学);UC Berkeley(加州大学伯克利分校);Robotics at Google(谷歌机器人部门)

Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, Chelsea Finn

摘要:

虽然深度学习和深度强化学习(RL)系统在图像分类、游戏和机器人控制等领域取得了令人印象深刻的成果,但数据效率仍然是一个重大挑战。多任务学习已经成为一种很有前途的方法,用于在多个任务之间共享结构,以实现更有效的学习。然而,多任务设置提出了许多优化挑战,使得与独立学习任务相比难以实现大的效率增益。与单任务学习相比,多任务学习如此具有挑战性的原因尚不完全清楚。在这项工作中,我们确定了一组三个条件的多任务优化景观,造成有害的梯度干扰,并开发了一个简单而通用的方法来避免这种干扰之间的任务梯度。我们提出了一种形式的梯度手术,将任务的梯度投影到具有冲突梯度的任何其他任务的梯度的法向平面上。在一系列具有挑战性的多任务监督和多任务强化学习问题上,这种方法在效率和性能上都有很大的提高。此外,它是模型不可知的,可以与先前提出的多任务架构相结合,以提高性能。

英文摘要:

While deep learning and deep reinforcement learning (RL) systems have demonstrated impressive results in domains such as image classification, game playing, and robotic control, data efficiency remains a major challenge. Multi-task learning has emerged as a promising approach for sharing structure across multiple tasks to enable more efficient learning. However, the multi-task setting presents a number of optimization challenges, making it difficult to realize large efficiency gains compared to learning tasks independently. The reasons why multi-task learning is so challenging compared to single-task learning are not fully understood. In this work, we identify a set of three conditions of the multi-task optimization landscape that cause detrimental gradient interference, and develop a simple yet general approach for avoiding such interference between task gradients. We propose a form of gradient surgery that projects a task's gradient onto the normal plane of the gradient of any other task that has a conflicting gradient. On a series of challenging multi-task supervised and multi-task RL problems, this approach leads to substantial gains in efficiency and performance. Further, it is model-agnostic and can be combined with previously-proposed multi-task architectures for enhanced performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑