Recent work has shown that pre-trained language models such as BERT improve robustness to spurious correlations in the dataset. Intrigued by these results, we find that the key to their success is generalization from a small amount of counterexamples where the spurious correlations do not hold. When such minority examples are scarce, pre-trained models perform as poorly as models trained from scratch. In the case of extreme minority, we propose to use multi-task learning (MTL) to improve generalization. Our experiments on natural language inference and paraphrase identification show that MTL with the right auxiliary tasks significantly improves performance on challenging examples without hurting the in-distribution performance. Further, we show that the gain from MTL mainly comes from improved generalization from the minority examples. Our results highlight the importance of data diversity for overcoming spurious correlations.
Quantifying and Improving the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Data
Simon Fraser University（西蒙弗雷泽大学）；Microsoft（微软）；Atlassian；Tongji University（同济大学）；The University of Hong Kong（香港大学）；Canada-CIFAR AI Chair, Amii（加拿大-CIFAR人工智能主席，Amii）
Beyond Spurious Signals: Debiasing Multimodal Large Language Models via Counterfactual Inference and Adaptive Expert Routing
School of Computer Science, Peking University（北京大学计算机科学学院）；MOE Key Laboratory of Computational Linguistics, Peking University（北京大学教育部计算语言学重点实验室）；National Key Laboratory for Multimedia Information Processing, Peking University（北京大学国家多媒体信息处理重点实验室）
Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoning
NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所NLPR与MAIS实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Meituan（美团）；University of Science and Technology of China（中国科学技术大学）
Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language
Google Research（谷歌研究院）
伪相关性是对自然语言处理系统可信度的威胁，促使人们研究识别和消除伪相关性的方法。然而，解决虚假相关的问题需要更清楚地了解它们是什么以及它们如何在语言数据中出现。Gardner et al（2021）认为，语言的组成性质意味着标签和个体“输入特征”之间的\emph{all}相关性是虚假的。本文分析了这一建议的背景下，一个玩具的例子，展示了三个不同的条件，可以引起一个简单的PCFG的特征标签的相关性。将玩具示例链接到结构化因果模型表明：（1）即使标签对特征上的干预是不变的，特征-标签相关性也会出现;（2）即使标签对特征上的干预敏感，特征-标签相关性也可能不存在。由于输入特征将单独与标签相关，但在非常罕见的情况下，必须应用领域知识来识别虚假的相关性，构成真正的鲁棒性威胁。
Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing
TomTom（TomTom公司）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；Saarland Informatics Campus（萨尔信息学园区）
CoBA: Counterbias Text Augmentation for Mitigating Various Spurious Correlations via Semantic Triples
DATUMO；AITRICS；Brainventures；Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University（Chung-Ang大学高级影像科学、多媒体与电影研究生院）；Department of Artificial Intelligence, Chung-Ang University（Chung-Ang大学人工智能系）
