描述和避免负迁移
Characterizing and Avoiding Negative Transfer
摘要:
当特定目标任务的标记数据稀缺时,迁移学习通常通过利用来自相关源任务的数据来提供有效的解决方案。然而,当从一个不太相关的来源转移知识时,它可能会反过来损害目标的表现,这种现象被称为负迁移。尽管负迁移现象普遍存在,但人们对它的描述往往是非正式的,缺乏严格的定义、细致的分析和系统的处理。本文提出了负迁移的正式定义,并分析了负迁移的三个重要方面。基于此分析,提出了一种新的技术,通过过滤掉不相关的源数据来规避负迁移。基于对抗网络,该技术具有高度通用性,可应用于广泛的迁移学习算法。通过在四个不同难度的基准数据集上进行实验,对六种最先进的深度传输方法进行了评估。经验上,所提出的方法一致地提高了所有基线方法的性能,并在很大程度上避免了负传递,即使当源数据是退化的。
英文摘要:
When labeled data is scarce for a specific target task, transfer learning often offers an effective solution by utilizing data from a related source task. However, when transferring knowledge from a less related source, it may inversely hurt the target performance, a phenomenon known as negative transfer. Despite its pervasiveness, negative transfer is usually described in an informal manner, lacking rigorous definition, careful analysis, or systematic treatment. This paper proposes a formal definition of negative transfer and analyzes three important aspects thereof. Stemming from this analysis, a novel technique is proposed to circumvent negative transfer by filtering out unrelated source data. Based on adversarial networks, the technique is highly generic and can be applied to a wide range of transfer learning algorithms. The proposed approach is evaluated on six state-of-the-art deep transfer methods via experiments on four benchmark datasets with varying levels of difficulty. Empirically, the proposed method consistently improves the performance of all baseline methods and largely avoids negative transfer, even when the source data is degenerate.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。