arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.08006cs.LGcs.AI
收藏

为多任务学习注入不平衡敏感性

Injecting Imbalance Sensitivity for Multi-Task Learning

University of Science and Technology of China(中国科学技术大学);Tencent AI Lab(腾讯AI实验室)

Zhipeng Zhou, Liu Liu, Peilin Zhao, Wei Gong

AI总结:

本文针对多任务学习中常被忽视的不平衡/主导性问题,提出不平衡敏感梯度下降方法IMGrad,通过约束投影范数增强基线方法,并在多个监督学习与强化学习基准上验证了其竞争力。

摘要:

多任务学习(MTL)已成为在现实应用中部署深度学习模型的一种有前景的方法。近期研究提出了基于优化的学习范式,以在MTL中建立任务共享表示。然而,本文通过实证论证指出,这些研究——特别是基于梯度的研究——主要强调冲突问题,而忽视了MTL中可能更为显著的不平衡/主导性影响。基于这一观点,我们通过对投影范数施加约束来注入不平衡敏感性,从而增强现有的基线方法。为证明我们提出的不平衡敏感梯度(IMGrad)下降方法的有效性,我们在多个主流MTL基准上对其进行了评估,涵盖监督学习任务和强化学习。实验结果一致表明其具有竞争力的性能。

英文摘要:

Multi-task learning (MTL) has emerged as a promising approach for deploying deep learning models in real-life applications. Recent studies have proposed optimization-based learning paradigms to establish task-shared representations in MTL. However, our paper empirically argues that these studies, specifically gradient-based ones, primarily emphasize the conflict issue while neglecting the potentially more significant impact of imbalance/dominance in MTL. In line with this perspective, we enhance the existing baseline method by injecting imbalance-sensitivity through the imposition of constraints on the projected norms. To demonstrate the effectiveness of our proposed IMbalance-sensitive Gradient (IMGrad) descent method, we evaluate it on multiple mainstream MTL benchmarks, encompassing supervised learning tasks as well as reinforcement learning. The experimental results consistently demonstrate competitive performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑