arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.13568cs.AI
收藏

LLMs信任谁?人类专家比其他LLMs更重要

Who Do LLMs Trust? Human Experts Matter More Than Other LLMs

Department of Computer Science, Indiana University Bloomington(计算机科学系,印第安纳大学布卢明顿分校);Department of Psychological and Brain Science, Indiana University Bloomington(心理学与脑科学系,印第安纳大学布卢明顿分校)

Anooshka Bajaj, Zoran Tiganj

AI总结:

本文研究LLMs在面对人类专家与其他LLMs反馈时的可信度影响,发现LLMs更倾向于信任人类专家的反馈。

摘要:

大型语言模型(LLMs)越来越多地在环境中运作,这些环境中会遇到社会信息,如其他代理的答案、工具输出或人类建议。在人类中,此类输入会影响判断的方式取决于来源的可信度和共识的强度。本文研究LLMs是否表现出类似的影响模式,以及它们是否优先考虑来自人类的反馈而非其他LLMs的反馈。在三个二元决策任务中,阅读理解、多步推理和道德判断,我们向四个经过指令微调的LLMs提供了先前响应,这些响应被归因于朋友、人类专家或其他LLMs。我们操纵群体是否正确,并改变群体大小。在第二个实验中,我们引入了单个人类和单个LLM之间的直接分歧。在各个任务中,模型更倾向于遵循被标记为来自人类专家的响应,包括当该信号错误时,并且比其他LLMs更容易修改答案以适应专家。这些结果揭示了专家框架作为当代LLMs的强先验,表明了一种在不同决策领域中普遍存在的基于可信度的社会影响形式。

英文摘要:

Large language models (LLMs) increasingly operate in environments where they encounter social information such as other agents' answers, tool outputs, or human recommendations. In humans, such inputs influence judgments in ways that depend on the source's credibility and the strength of consensus. This paper investigates whether LLMs exhibit analogous patterns of influence and whether they privilege feedback from humans over feedback from other LLMs. Across three binary decision-making tasks, reading comprehension, multi-step reasoning, and moral judgment, we present four instruction-tuned LLMs with prior responses attributed either to friends, to human experts, or to other LLMs. We manipulate whether the group is correct and vary the group size. In a second experiment, we introduce direct disagreement between a single human and a single LLM. Across tasks, models conform significantly more to responses labeled as coming from human experts, including when that signal is incorrect, and revise their answers toward experts more readily than toward other LLMs. These results reveal that expert framing acts as a strong prior for contemporary LLMs, suggesting a form of credibility-sensitive social influence that generalizes across decision domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑