Transfer learning (TL) utilizes data or knowledge from one or more source domains to facilitate the learning in a target domain. It is particularly useful when the target domain has very few or no labeled data, due to annotation expense, privacy concerns, etc. Unfortunately, the effectiveness of TL is not always guaranteed. Negative transfer (NT), i.e., leveraging source domain data/knowledge undesirably reduces the learning performance in the target domain, has been a long-standing and challenging problem in TL. Various approaches have been proposed in the literature to handle it. However, there does not exist a systematic survey on the formulation of NT, the factors leading to NT, and the algorithms that mitigate NT. This paper fills this gap, by first introducing the definition of NT and its factors, then reviewing about fifty representative approaches for overcoming NT, according to four categories: secure transfer, domain similarity estimation, distant transfer, and NT mitigation. NT in related fields, e.g., multi-task learning, lifelong learning, and adversarial attacks, are also discussed.
Mitigating Negative Transfer via Reducing Environmental Disagreement
National Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学);School of Artificial Intelligence, Nanjing University(人工智能学院,南京大学)
Which Model to Transfer? A Survey on Transferability Estimation
School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);Institute of Information Engineering, CAS(中国科学院信息工程研究所);University of Chinese Academy of Sciences(中国科学院大学);School of Artificial Intelligence, Anhui University(安徽大学人工智能学院);CRIPAC & MAIS, Institute of Automation, Chinese Academy of Sciences (CAS)(中国科学院自动化研究所模式识别国家重点实验室与多模态人工智能系统实验室)
A Survey of Incremental Transfer Learning: Combining Peer-to-Peer Federated Learning and Domain Incremental Learning for Multicenter Collaboration
University Hospital Erlangen(埃尔朗根大学医院);Friedrich-Alexander-Universität Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);Comprehensive Cancer Center Erlangen-EMN(埃尔朗根-EMN综合癌症中心);Bavarian Cancer Research Center(巴伐利亚癌症研究中心)
Understanding Knowledge Transferability for Transfer Learning: A Survey
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Shenzhen Institute of Advanced Technology Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
Proxy-informed Bayesian transfer learning with unknown sources
Department of Computer Science University of Manchester(曼彻斯特大学计算机科学系);Inserm Bordeaux Population Health, Vaccine Research Institute, Université de Bordeaux, Inria Bordeaux Sud-ouest(法国波尔多大学疫苗研究学院);Department of Computer Science Aalto University(埃斯波芬兰阿尔托大学计算机科学系);University of Manchester(曼彻斯特大学);Aalto University(阿尔托大学);ELLIS Institute Finland(芬兰ELLIS研究所)
When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey
East China University of Science and Technology(华东理工大学);Oklahoma State University(俄克拉荷马州立大学);Potsdam Institute for Climate Impact Research(波茨坦气候影响研究所);Humboldt University of Berlin(柏林洪堡大学)
A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications
Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能研究院);School of Computer Science and Technology, Shandong University(山东大学计算机科学与技术学院);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);Beijing Academy of Blockchain and Edge Computing(北京区块链与边缘计算研究院)
A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions
Centre for Artificial Intelligence(人工智能中心);ZHAW School of Engineering(苏黎世应用科技大学工程学院);Institute of Embedded Systems(嵌入式系统研究所);Kistler Instrumente AG(奇石乐仪器股份公司);Institute of Neuroinformatics(神经信息学研究所);ETH & University of Zürich(苏黎世联邦理工学院与苏黎世大学);ZHAW Centre for Artificial Intelligence(苏黎世应用科技大学人工智能中心);European Centre for Living Technology (ECLT)(欧洲活技术中心(ECLT))
Bayesian Meta-Learning with Expert Feedback for Task-Shift Adaptation through Causal Embeddings
Department of Computer Science, Aalto University, Espoo, Finland(Aalto大学计算机科学系);ELLIS Institute Finland, Espoo, Finland(芬兰ELLIS研究所);Department of Computer Science, University of Manchester, United Kingdom(曼彻斯特大学计算机科学系)
Overcoming Negative Transfer by Online Selection: Distant Domain Adaptation for Fault Diagnosis
School of Computer Science and Engineering, Nanyang Technological University(南洋理工大学计算机科学与工程学院);Institute for Infocomm Research, Agency for Science, Technology and Research(新加坡科技研究局信息通信研究院);Advanced Institute of Natural Sciences, Beijing Normal University(北京师范大学自然科学高等研究院);Department of Computer Science and Engineering, The Chinese University of Hong Kong(香港中文大学计算机科学与工程学系)
Transfer Learning for Transient Classification: From Simulations to Real Data and ZTF to LSST
Kavli Institute for Astrophysics and Space Research, Massachusetts Institute of Technology(凯斯勒天体物理与空间研究研究所,麻省理工学院);Irvington High School(伊里弗顿高中);Department of Physics and Astronomy, Northwestern University(物理与天文学系,西北大学);Center for Interdisciplinary Exploration and Research in Astrophysics (CIERA)(天体物理学跨学科探索与研究中心(CIERA));NSF-Simons AI Institute for the Sky (SkAI)(NSF-模拟ons人工智能研究所(SkAI))
MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);Max Planck Institute(马克斯·普朗克研究所);ETH Zürich(苏黎世联邦理工学院);UC San Diego(加州大学圣迭戈分校);University of Waterloo(滑铁卢大学);University of Michigan(密歇根大学)
Reset & Distill: A Recipe for Overcoming Negative Transfer in Continual Reinforcement Learning
Department of Electrical and Computer Engineering (ECE), Seoul National University(电子与计算机工程系,首尔国立大学);Interdisciplinary Program in Artificial Intelligence (IPAI), Seoul National University(人工智能交叉计划,首尔国立大学);Samsung Advanced Institute of Technology(三星先进技术研究所)