LLMs are increasingly used for high-stakes decision-making, yet their sensitivity to spurious context can introduce harmful biases. This is a critical concern when models are deployed for tasks like evaluating teachers' instructional quality, where biased assessment can affect teachers' professional development and career. We investigate model robustness to spurious social contexts about teachers using the largest publicly available dataset of U.S. classroom transcripts (NCTE) paired with expert evaluation scores. Evaluating seven frontier and open-weight models across seven categories of spurious contexts -- including teacher experience, education level, demographic identity, and sycophancy-inducing framings -- we find that irrelevant contexts can shift model-generated ratings by up to 1.48 points on a 7-point scale. Prompt-based mitigations and popular post-training methods, such as Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO), fall short of addressing the issue. We propose Debiasing-DPO, which combines contrastive reasoning-augmented DPO with SFT on expert labels to reduce the effect of spurious context while avoiding mode collapse. Applied to Llama and Qwen Instruct models of 3-8B parameters, Debiasing-DPO reduces bias by 84% and improves predictive accuracy by 52% on average across models. Our findings from the educational dataset highlight that stronger models may exhibit greater sensitivity despite higher accuracy, and Debiasing-DPO can improve both accuracy and robustness in prompt-based prediction tasks.
Spurious Correlations and Beyond: Understanding and Mitigating Shortcut Learning in SDOH Extraction with Large Language Models
Department of Computer Science(计算机科学系);School of Nursing(护理学院);Department of Information Sciences and Technology(信息科学与技术系);Department of Biomedical Informatics & Medical Education(生物医学信息学与医学教育系)
Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoning
NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所NLPR与MAIS实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Meituan(美团);University of Science and Technology of China(中国科学技术大学)
Mitigating Biases of Large Language Models in Stance Detection with Counterfactual Augmented Calibration
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Peng Cheng Laboratory(鹏城实验室);The Chinese University of Hong Kong(香港中文大学);King’s College London(伦敦国王学院);The 30th Research Institute of China Electronics Technology Group Corporation(中国电子科技集团公司第三十研究所)
Mitigating Shortcuts in Language Models with Soft Label Encoding
Stevens Institute of Technology(史蒂文斯理工学院);Shanghai Jiao Tong University(上海交通大学);New Jersey Institute of Technology(新泽西理工学院);University of Georgia(佐治亚大学)
Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective
Shanghai AI Laboratory(上海人工智能实验室);State Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室);School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);School of Computer Science, Fudan University(复旦大学计算机科学技术学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Evaluating and Mitigating LLM-as-a-judge Bias in Communication Systems
School of Computer Science and Engineering at Nanyang Technological University(南洋理工大学计算机科学与工程学院);School of Cyber Science and Engineering, Wuhan University(武汉大学网络科学与工程学院)
LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs
National University of Singapore(新加坡国立大学);VinAI Research(VinAI研究院);Institute of High Performance Computing (IHPC), A*STAR(新加坡科技研究局高性能计算研究所);Salesforce Research(Salesforce研究院);Nanyang Technological University(南洋理工大学)
Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Peking University(北京大学);Tencent Inc(腾讯公司)
CoBA: Counterbias Text Augmentation for Mitigating Various Spurious Correlations via Semantic Triples
DATUMO;AITRICS;Brainventures;Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University(Chung-Ang大学高级影像科学、多媒体与电影研究生院);Department of Artificial Intelligence, Chung-Ang University(Chung-Ang大学人工智能系)
Exposing and Mitigating Calibration Biases and Demographic Unfairness in MLLM Few-Shot In-Context Learning for Medical Image Classification
Centre for Intelligent Machines, McGill University, Montreal, Canada(智能机器中心,麦吉尔大学,加拿大);Mila -- Quebec AI Institute, Montreal, Canada(魁北克人工智能研究所);Stanford University, Stanford, USA(斯坦福大学);University of Copenhagen, Copenhagen, Denmark(哥本哈根大学)