通过经济游戏 eliciting 大语言模型的信任度先验
Eliciting Trustworthiness Priors of Large Language Models via Economic Games
AI总结:
通过经济游戏实验,研究如何通过行为博弈论中的信任游戏获取大语言模型的信任度先验,并揭示其与人类信任差异及刻板印象模型的关系。
摘要:
构建以人为本、可信的人工智能系统的一个关键方面是维持校准的信任:适当依赖AI系统优于过度信任(例如,自动化偏差)和不足信任(例如,不使用)。然而,一个根本性的挑战是如何表征AI系统自身所表现出的信任水平。在这里,我们提出了一种基于迭代上下文学习(Zhu和Griffiths,2024a)的新方法,并将其应用于通过行为博弈论中的信任游戏来获取信任度先验。信任游戏特别适合此目的,因为它将信任定义为基于对另一个代理的信念而自愿承担风险,而不是自我报告的态度。使用我们的方法,我们从几个领先的大型语言模型(LLMs)中获取了信任度先验,并发现GPT-4.1的信任度先验紧密跟踪人类观察到的那些。基于这一结果,我们进一步研究了GPT-4.1在信任游戏中对不同玩家人设的响应,提供了对这些模型如何在代理特征上区分信任的初步描述。最后,我们显示,所获取的信任度的变异可以很好地通过基于刻板印象的模型来预测,该模型基于感知的温暖和能力。
英文摘要:
One critical aspect of building human-centered, trustworthy artificial intelligence (AI) systems is maintaining calibrated trust: appropriate reliance on AI systems outperforms both overtrust (e.g., automation bias) and undertrust (e.g., disuse). A fundamental challenge, however, is how to characterize the level of trust exhibited by an AI system itself. Here, we propose a novel elicitation method based on iterated in-context learning (Zhu and Griffiths, 2024a) and apply it to elicit trustworthiness priors using the Trust Game from behavioral game theory. The Trust Game is particularly well suited for this purpose because it operationalizes trust as voluntary exposure to risk based on beliefs about another agent, rather than self-reported attitudes. Using our method, we elicit trustworthiness priors from several leading large language models (LLMs) and find that GPT-4.1's trustworthiness priors closely track those observed in humans. Building on this result, we further examine how GPT-4.1 responds to different player personas in the Trust Game, providing an initial characterization of how such models differentiate trust across agent characteristics. Finally, we show that variation in elicited trustworthiness can be well predicted by a stereotype-based model grounded in perceived warmth and competence.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。