Deep neural networks (DNNs) are vulnerable to shortcut learning: rather than learning the intended task, they tend to draw inconclusive relationships between their inputs and outputs. Shortcut learning is ubiquitous among many failure cases of neural networks, and traces of this phenomenon can be seen in their generalizability issues, domain shift, adversarial vulnerability, and even bias towards majority groups. In this paper, we argue that this commonality in the cause of various DNN issues creates a significant opportunity that should be leveraged to find a unified solution for shortcut learning. To this end, we outline the recent advances in topological data analysis (TDA), and persistent homology (PH) in particular, to sketch a unified roadmap for detecting shortcuts in deep learning. We demonstrate our arguments by investigating the topological features of computational graphs in DNNs using two cases of unlearnable examples and bias in decision-making as our test studies. Our analysis of these two failure cases of DNNs reveals that finding a unified solution for shortcut learning in DNNs is not out of reach, and TDA can play a significant role in forming such a framework.
ClearGCD: Mitigating Shortcut Learning For Robust Generalized Category Discovery
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Loughborough University（洛桑大学）
DynaCF: Mitigating Shortcut Learning in Reward Models via Dynamic Counterfactual Sensitivity
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；New Jersey Institute of Technology（新泽西理工学院）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）
Mitigating Backdoors via Decoy Shortcuts and Knowledge Decoupling
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Mitigating Shortcut Learning with InterpoLated Learning
Department of Computer Science and Technology, University of Cambridge（计算机科学与技术系，剑桥大学）；Department of Engineering, University of Cambridge（工程系，剑桥大学）；The Alan Turing Institute（艾伦·图灵研究所）
Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensembles
Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；Eberhard-Karls-Universität Tübingen（埃伯哈德卡尔斯蒂宾根大学）；Idiap Research Institute（伊迪亚普研究所）
MiMu: Mitigating Multiple Shortcut Learning Behavior of Transformers
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China（认知智能国家重点实验室，中国科学技术大学）；Hefei University of Technology（合肥工业大学）；iFLYTEK AI Research (Central China), iFLYTEK Co., Ltd（iFLYTEK中央研究院，iFLYTEK公司）
Mitigating Shortcuts in Language Models with Soft Label Encoding
Stevens Institute of Technology（史蒂文斯理工学院）；Shanghai Jiao Tong University（上海交通大学）；New Jersey Institute of Technology（新泽西理工学院）；University of Georgia（佐治亚大学）
Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets
Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Technical University of Munich（慕尼黑工业大学）；IEO European Institute of Oncology IRCCS（IEO欧洲肿瘤研究所IRCCS）
Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency
Sydeny AI Centre（悉尼人工智能中心）；School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；Hong Kong Baptist University（香港浸会大学）；BNRist Center, Tsinghua University（清华大学BNRist中心）
Improved physics-informed neural network in mitigating gradient related failures
College of Applied Mathematics, Chengdu University of Information Technology（成都信息工程大学应用数学学院）；College of Mathematics, Sichuan University（四川大学数学学院）；School of Sciences, Southwest Petroleum University（西南石油大学理学院）
