Drawing on constructs from psychology, prior work has identified a distinction between explicit and implicit bias in large language models (LLMs). While many LLMs undergo post-training alignment and safety procedures to avoid expressions of explicit social bias, they still exhibit significant implicit biases on indirect tasks resembling the Implicit Association Test (IAT). Recent work has further shown that inference-time reasoning can impair LLM performance on tasks that rely on implicit statistical learning. Motivated by a theoretical link between implicit associations and statistical learning in human cognition, we examine how reasoning-enabled inference affects implicit bias in LLMs. We find that enabling reasoning significantly reduces measured implicit bias on an IAT-style evaluation for some model classes across fifteen stereotype topics. This effect appears specific to social bias domains, as we observe no corresponding reduction for non-social implicit associations. As reasoning is increasingly enabled by default in deployed LLMs, these findings suggest that it can meaningfully alter fairness evaluation outcomes in some systems, while also raising questions about how alignment procedures interact with inference-time reasoning to drive variation in bias reduction across model types. More broadly, this work highlights how theory from cognitive science and psychology can complement AI evaluation research by providing methodological and interpretive frameworks that reveal new insights into model behavior.
Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflection
College of Intelligence and Computing, Tianjin University(智能与计算学院,天津大学);AI Lab, China Mobile Communication Group Tianjin Co., Ltd.(中国移动通信集团天津有限公司人工智能实验室)
SOCIALITE-LLAMA: An Instruction-Tuned Model for Social Scientific Tasks
Stony Brook University(石溪大学);JP Morgan Chase(摩根大通);Bloomberg(彭博公司);Preferabli(普雷费布利公司);University of Pennsylvania(宾夕法尼亚大学);National Institute of Health(美国国立卫生研究院);Grammarly(grammarly公司)
Two-Stage Voting for Robust and Efficient Suicide Risk Detection on Social Media
Department of Electrical and Computer Engineering(电气与计算机工程系);Department of Informatics and Networked Systems(信息与网络系统系);University of Pittsburgh(匹兹堡大学);Department of Psychiatry(精神病学系);Tandon School of Engineering(塔能工程学院);New York University(纽约大学)
A word association network methodology for evaluating implicit biases in LLMs compared to humans
University of Pisa, Department of Computer Science(比萨大学计算机科学系);National Research Council of Italy, Institute of Information Science and Technologies(意大利国家研究委员会信息科学与技术研究所);University of Trento, Department of Psychology and Cognitive Science(特伦托大学心理学与认知科学系)
MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Domain Risks in LLMs
School of Computer Science and Technology, East China Normal University(东华大学计算机科学与技术学院);Shanghai AI Lab, Shanghai Innovation Institute(上海人工智能实验室,上海创新研究院)
MuSeD: A Multimodal Spanish Dataset for Sexism Detection in Social Media Videos
University of Barcelona, CLiC-Language and Computing Center(巴塞罗那大学,CLiC语言与计算中心);University of Copenhagen, Department of Computer Science(哥本哈根大学,计算机科学系)
Analyzing Cognitive Differences Among Large Language Models through the Lens of Social Worldview
Wangxuan Institute of Computer Technology, Peking University(北京大学计算机技术研究院);Information Management Department, Peking University(北京大学信息管理系);Renmin University of China(中国人民大学)
Same Values, Different Languages? From Multilingual Probing to Steering LLMs Toward Chinese Social Values
School of Information Science and Technology, Beijing Foreign Studies University(北京外国语大学信息科学与技术学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Beihang University(北京航空航天大学)
Detecting Early and Implicit Suicidal Ideation via Longitudinal and Information Environment Signals on Social Media
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Columbia Irving Medical Center(哥伦比亚大学伊万尼兹医疗中心);New York State Psychiatric Institute(纽约州精神医学研究所)