Who Do LLMs Trust? Human Experts Matter More Than Other LLMs
Department of Computer Science, Indiana University Bloomington(计算机科学系,印第安纳大学布卢明顿分校);Department of Psychological and Brain Science, Indiana University Bloomington(心理学与脑科学系,印第安纳大学布卢明顿分校)
Large language models (LLMs) increasingly operate in environments where they encounter social information such as other agents' answers, tool outputs, or human recommendations. In humans, such inputs influence judgments in ways that depend on the source's credibility and the strength of consensus. This paper investigates whether LLMs exhibit analogous patterns of influence and whether they privilege feedback from humans over feedback from other LLMs. Across three binary decision-making tasks, reading comprehension, multi-step reasoning, and moral judgment, we present four instruction-tuned LLMs with prior responses attributed either to friends, to human experts, or to other LLMs. We manipulate whether the group is correct and vary the group size. In a second experiment, we introduce direct disagreement between a single human and a single LLM. Across tasks, models conform significantly more to responses labeled as coming from human experts, including when that signal is incorrect, and revise their answers toward experts more readily than toward other LLMs. These results reveal that expert framing acts as a strong prior for contemporary LLMs, suggesting a form of credibility-sensitive social influence that generalizes across decision domains.
Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert-Like Systems Engineering Artifacts and a Characterization of Failure Modes
Virginia Tech(弗吉尼亚理工大学);Scale AI(Scale AI公司);Virginia Tech National Security Institute(弗吉尼亚理工大学国家安全研究所)
Can AI Extract Antecedent Factors of Human Trust in AI? An Application of Information Extraction for Scientific Literature in Behavioural and Computer Sciences
A Citation-Grounded Benchmark for Trustworthy Earnings Call Transcript Analysis with Large Language Models
American Express(美国运通);Singapore Decision Science Center of Excellence, American Express(美国运通新加坡决策科学卓越中心);Global Decision Science, American Express(美国运通全球决策科学部)
Building Trust in Mental Health Chatbots: Safety Metrics and LLM-Based Evaluation Tools
Sue & Bill Gross School of Nursing, University of California, Irvine(加州大学欧文分校苏与比尔·格罗斯护理学院);Department of Computer Sciences, University of California, Irvine(加州大学欧文分校计算机科学系);School of Medicine, University of California, Irvine(加州大学欧文分校医学院);Department of Psychological Science, University of California, Irvine(加州大学欧文分校心理科学系);Chicago School of Professional Psychology(芝加哥专业心理学院);HealthUnity
A Comparative Approach to Explainable Artificial Intelligence Methods in Application to High-Dimensional Electronic Health Records: Examining the Usability of XAI
Correct Yourself, Keep My Trust: How Self-Correction and Social Connection Shape Credibility in Social Chatbots
School of Computing National University of Singapore Singapore Singapore(计算学院新加坡国立大学新加坡新加坡);Computer Science National University of Singapore Singapore Singapore(计算机科学新加坡国立大学新加坡新加坡);National University of Singapore(新加坡国立大学)
Human-centered trust framework: An HCI perspective
School of digital technologies, Tallinn University(塔林大学数字技术学院);Universidade de Trás-os-Montes e Alto Douro(特拉斯-os-蒙特斯和上杜罗大学);INESC TEC(INESC TEC(工程系统与技术计算机研究所));CIDTFF(CIDTFF(人类与社会科学研究中心))