arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.11986cs.LG
收藏

多任务优化的选择性任务组更新

Selective Task Group Updates for Multi-Task Optimization

Korea Advanced Institute of Science and Technology(韩国科学技术院)

Wooseong Jeong, Kuk-Jin Yoon

AI总结:

针对多任务学习中的负迁移问题,提出通过选择性任务分组和顺序更新来增强任务特定参数学习,并引入近端任务间亲和度进行优化,显著提升多任务性能。

摘要:

多任务学习通过在统一架构中训练多个任务,能够获取任务通用知识。然而,在单一架构中同时训练所有任务可能导致性能下降,即负迁移,这是多任务学习中的一个主要关注点。以往的工作通过梯度操作或加权损失调整来优化多任务网络,以解决这一问题。然而,它们的优化策略侧重于解决共享参数中的任务不平衡,忽略了任务特定参数的学习。因此,它们在缓解负迁移方面表现出局限性,因为在优化过程中,共享空间和任务特定信息的学习相互影响。为了解决这一问题,我们提出了一种不同的方法,通过在优化过程中对每个批次的任务进行选择性分组和更新,来增强多任务性能。我们引入了一种算法,能够自适应地决定如何在训练过程中有效地对任务进行分组和更新。为了同时跟踪任务间关系并优化多任务网络,我们提出了近端任务间亲和度,该指标可以在优化过程中进行测量。我们提供了理论分析,说明将任务划分为多个组并顺序更新,通过增强任务特定参数的学习,显著影响多任务性能。我们的方法大幅优于以往的多任务优化方法,并且可扩展到不同架构和不同数量的任务。

英文摘要:

Multi-task learning enables the acquisition of task-generic knowledge by training multiple tasks within a unified architecture. However, training all tasks together in a single architecture can lead to performance degradation, known as negative transfer, which is a main concern in multi-task learning. Previous works have addressed this issue by optimizing the multi-task network through gradient manipulation or weighted loss adjustments. However, their optimization strategy focuses on addressing task imbalance in shared parameters, neglecting the learning of task-specific parameters. As a result, they show limitations in mitigating negative transfer, since the learning of shared space and task-specific information influences each other during optimization. To address this, we propose a different approach to enhance multi-task performance by selectively grouping tasks and updating them for each batch during optimization. We introduce an algorithm that adaptively determines how to effectively group tasks and update them during the learning process. To track inter-task relations and optimize multi-task networks simultaneously, we propose proximal inter-task affinity, which can be measured during the optimization process. We provide a theoretical analysis on how dividing tasks into multiple groups and updating them sequentially significantly affects multi-task performance by enhancing the learning of task-specific parameters. Our methods substantially outperform previous multi-task optimization approaches and are scalable to different architectures and various numbers of tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑