arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.00288cs.CVcs.AIcs.LG
收藏

无监督域自适应中的梯度协调

Gradient Harmonization in Unsupervised Domain Adaptation

School of Microelectronics and Communication Engineering, Chongqing University(重庆大学微电子与通信工程学院)

Fuxiang Huang, Suqi Song, Lei Zhang

AI总结:

本文针对无监督域自适应中域对齐与分类任务的梯度冲突问题,提出梯度协调方法 GH 和 GH++,通过调整任务间梯度夹角并转化为动态加权损失函数,在提升现有 UDA 基线及最新模型性能的同时保持方法正交可集成。

摘要:

无监督域自适应(UDA)旨在将知识从有标签的源域迁移到无标签的目标域。当前许多方法专注于通过同时优化域对齐和分类任务,来学习既对分类具有判别性又跨域不变的特征表示。然而,这些方法往往忽视了一个关键挑战:在基于梯度的优化过程中,这两个任务之间存在固有冲突。本文深入探讨了这一问题,并提出了两种有效的解决方案,统称为梯度协调(Gradient Harmonization),包括 GH 和 GH++,以缓解域对齐与分类任务之间的冲突。GH 的作用是将不同任务之间的梯度夹角从钝角改变为锐角,从而解决冲突并以协调的方式权衡这两个任务。然而,这会导致两个任务都偏离其原始优化方向。因此,我们进一步提出了改进版本 GH++,它将任务之间的梯度夹角从钝角调整为直角。这不仅消除了冲突,还最大限度地减少了对原始梯度方向的偏离。最后,为了优化方便和效率,我们利用协调后梯度上的积分算子,将梯度协调策略演化为一个动态加权损失函数。值得注意的是,GH/GH++ 与 UDA 是正交的,可以无缝集成到大多数现有的 UDA 模型中。理论分析和实验分析表明,所提出的方法不仅增强了流行的 UDA 基线模型,还改进了近期最先进的模型。

英文摘要:

Unsupervised domain adaptation (UDA) intends to transfer knowledge from a labeled source domain to an unlabeled target domain. Many current methods focus on learning feature representations that are both discriminative for classification and invariant across domains by simultaneously optimizing domain alignment and classification tasks. However, these methods often overlook a crucial challenge: the inherent conflict between these two tasks during gradient-based optimization. In this paper, we delve into this issue and introduce two effective solutions known as Gradient Harmonization, including GH and GH++, to mitigate the conflict between domain alignment and classification tasks. GH operates by altering the gradient angle between different tasks from an obtuse angle to an acute angle, thus resolving the conflict and trade-offing the two tasks in a coordinated manner. Yet, this would cause both tasks to deviate from their original optimization directions. We thus further propose an improved version, GH++, which adjusts the gradient angle between tasks from an obtuse angle to a vertical angle. This not only eliminates the conflict but also minimizes deviation from the original gradient directions. Finally, for optimization convenience and efficiency, we evolve the gradient harmonization strategies into a dynamically weighted loss function using an integral operator on the harmonized gradient. Notably, GH/GH++ are orthogonal to UDA and can be seamlessly integrated into most existing UDA models. Theoretical insights and experimental analyses demonstrate that the proposed approaches not only enhance popular UDA baselines but also improve recent state-of-the-art models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑