arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.02585cs.AIcs.CL
收藏

通过直接偏好优化减轻LLM对虚假社会情境的偏见

Mitigating LLM biases toward spurious social contexts using direct preference optimization

Stanford University(斯坦福大学)

Hyunji Nam, Dorottya Demszky

AI总结:

本文研究了LLM对虚假社会情境的鲁棒性,提出Debiasing-DPO方法,通过自监督训练和监督微调减少偏见并提升预测准确性。

摘要:

LLM在高风险决策中日益普及,但对虚假上下文的敏感性可能引入有害偏见。本文利用美国课堂 transcripts 数据集(NCTE)和专家评分,评估七种前沿模型在七类虚假情境(如教师经验、教育水平、人口特征和谄媚性框架)中的表现。结果显示,无关上下文信息可能使模型预测偏离高达1.48个点(7分制),大模型有时更敏感。通过提示和标准DPO缓解不足,本文提出Debiasing-DPO方法,结合中性推理与偏见推理,结合监督微调提升准确性。应用于Llama 3B/8B和Qwen 3B/7B Instruct模型,Debiasing-DPO减少偏见84%,提升预测准确性52%。教育案例研究显示,模型鲁棒性并非自然由规模提升产生,本文方法在提示预测任务中显著提升准确性和鲁棒性。

英文摘要:

LLMs are increasingly used for high-stakes decision-making, yet their sensitivity to spurious context can introduce harmful biases. This is a critical concern when models are deployed for tasks like evaluating teachers' instructional quality, where biased assessment can affect teachers' professional development and career. We investigate model robustness to spurious social contexts about teachers using the largest publicly available dataset of U.S. classroom transcripts (NCTE) paired with expert evaluation scores. Evaluating seven frontier and open-weight models across seven categories of spurious contexts -- including teacher experience, education level, demographic identity, and sycophancy-inducing framings -- we find that irrelevant contexts can shift model-generated ratings by up to 1.48 points on a 7-point scale. Prompt-based mitigations and popular post-training methods, such as Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO), fall short of addressing the issue. We propose Debiasing-DPO, which combines contrastive reasoning-augmented DPO with SFT on expert labels to reduce the effect of spurious context while avoiding mode collapse. Applied to Llama and Qwen Instruct models of 3-8B parameters, Debiasing-DPO reduces bias by 84% and improves predictive accuracy by 52% on average across models. Our findings from the educational dataset highlight that stronger models may exhibit greater sensitivity despite higher accuracy, and Debiasing-DPO can improve both accuracy and robustness in prompt-based prediction tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑