A Too-Good-to-be-True Prior to Reduce Shortcut Reliance
HSE University（俄罗斯高等经济大学）；University College London（伦敦大学学院）；Research Center Jülich（尤利希研究中心）；Heinrich Heine University Düsseldorf（海因里希·海涅大学杜塞尔多夫分校）；The Alan Turing Institute（阿兰·图灵研究所）
Nikolay Dagaev, Brett D. Roads, Xiaoliang Luo, Daniel N. Barry, Kaustubh R. Patil, Bradley C. Love
Despite their impressive performance in object recognition and other tasks under standard testing conditions, deep networks often fail to generalize to out-of-distribution (o.o.d.) samples. One cause for this shortcoming is that modern architectures tend to rely on "shortcuts" - superficial features that correlate with categories without capturing deeper invariants that hold across contexts. Real-world concepts often possess a complex structure that can vary superficially across contexts, which can make the most intuitive and promising solutions in one context not generalize to others. One potential way to improve o.o.d. generalization is to assume simple solutions are unlikely to be valid across contexts and avoid them, which we refer to as the too-good-to-be-true prior. A low-capacity network (LCN) with a shallow architecture should only be able to learn surface relationships, including shortcuts. We find that LCNs can serve as shortcut detectors. Furthermore, an LCN's predictions can be used in a two-stage approach to encourage a high-capacity network (HCN) to rely on deeper invariant features that should generalize broadly. In particular, items that the LCN can master are downweighted when training the HCN. Using a modified version of the CIFAR-10 dataset in which we introduced shortcuts, we found that the two-stage LCN-HCN approach reduced reliance on shortcuts and facilitated o.o.d. generalization.
Close Shortcut Wins Long: Seeking Diverse and Stable Generators for Data-Free Knowledge Distillation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Zhongguancun Academy（中关村学院）；School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；Shandong Computer Science Center（山东省计算机科学中心）；Spatialtemporal AI（时空人工智能公司）；Tongji University（同济大学）
Efficient Unsupervised Shortcut Learning Detection and Mitigation in Transformers
Goethe Universität（法兰克福歌德大学）；Philipps-Universität（马尔堡菲利普大学）；German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；German Cancer Consortium (DKTK)（德国癌症联盟（DKTK））
ICON decomposition: Multivariate concept auditing reveals shortcuts that neural networks rely on
Hertie Institute for AI in Brain Health, University of Tübingen（蒂宾根大学赫蒂脑健康人工智能研究所）；Charité - Universitätsmedizin Berlin（柏林夏里特医学院）；Humboldt-Universität zu Berlin（柏林洪堡大学）；Tübingen AI Center, University of Tübingen（蒂宾根大学蒂宾根人工智能中心）；German Center for Mental Health (DZPG)（德国心理健康中心）；Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希赫兹研究所）；Technische Universität Berlin（柏林工业大学）
High-Order Matching for One-Step Shortcut Diffusion Models
Middle Tennessee State University（中田纳西州立大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of New South Wales（新南威尔士大学）；The University of Hong Kong（香港大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Tsinghua University（清华大学）；The Simons Institute for the Theory of Computing at UC Berkeley（加州大学伯克利分校西蒙斯理论计算研究所）；Anhui University（安徽大学）
Seeing Time: Benchmarking Chronological Reasoning and Shortcut Biases in Vision-Language Models
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computing Technologies, RMIT University（皇家墨尔本理工学院计算技术学院）
