大型语言模型是战略决策者吗？对两人非零和博弈中表现与偏差的研究
Are Large Language Models Strategic Decision Makers? A Study of Performance and Bias in Two-Player Non-Zero-Sum Games
AI总结：
本研究评估了GPT-3.5、GPT-4-Turbo、GPT-4o和Llama-3-8B在两人非零和博弈中的战略决策能力，发现它们受位置、收益和行为偏差影响，且CoT提示无法完全消除这些偏差。
摘要：
大型语言模型（LLMs）在现实世界环境中的应用日益增多，但其战略决策能力在很大程度上仍未被探索。为了充分受益于LLMs的潜力，了解它们在复杂社会场景中的运作能力至关重要。博弈论已被用于理解现实世界的互动，它为评估这些能力提供了一个良好的框架。本研究调查了LLMs在经典博弈论两人非零和博弈——Stag Hunt和Prisoner's Dilemma中的表现与优势。我们对GPT-3.5、GPT-4-Turbo、GPT-4o和Llama-3-8B进行的结构化评估表明，这些模型在这些游戏中做决策时，至少受到以下一种系统性偏差的影响：位置偏差、收益偏差或行为偏差。这表明LLMs在做出这些战略决策时并没有完全依赖逻辑推理。结果发现，当游戏配置与所受偏差不一致时，LLMs的表现会下降。在Stag Hunt中，当出现不一致时，GPT-3.5、GPT-4-Turbo、GPT-4o和Llama-3-8B的平均表现分别下降了32%、25%、34%和29%；而在Prisoner's Dilemma中分别下降了28%、16%、34%和24%。令人惊讶的是，GPT-4o（在标准基准测试中表现顶尖的LLM）遭受了最显著的表现下降，这表明较新的模型并未解决这些问题。有趣的是，我们发现一种常用于提升LLMs推理能力的方法——思维链（CoT）提示，减少了GPT-3.5、GPT-4o和Llama-3-8B中的偏差，但却增加了GPT-4-Turbo中的偏差影响，这表明仅靠CoT不能完全作为解决该问题的稳健方案。我们进行了几项额外的实验，为这些观察到的行为提供了进一步的见解。
英文摘要：
Large Language Models (LLMs) have been increasingly used in real-world settings, yet their strategic decision-making abilities remain largely unexplored. To fully benefit from the potential of LLMs, it's essential to understand their ability to function in complex social scenarios. Game theory, which is already used to understand real-world interactions, provides a good framework for assessing these abilities. This work investigates the performance and merits of LLMs in canonical game-theoretic two-player non-zero-sum games, Stag Hunt and Prisoner Dilemma. Our structured evaluation of GPT-3.5, GPT-4-Turbo, GPT-4o, and Llama-3-8B shows that these models, when making decisions in these games, are affected by at least one of the following systematic biases: positional bias, payoff bias, or behavioural bias. This indicates that LLMs do not fully rely on logical reasoning when making these strategic decisions. As a result, it was found that the LLMs' performance drops when the game configuration is misaligned with the affecting biases. When misaligned, GPT-3.5, GPT-4-Turbo, GPT-4o, and Llama-3-8B show an average performance drop of 32\%, 25\%, 34\%, and 29\% respectively in Stag Hunt, and 28\%, 16\%, 34\%, and 24\% respectively in Prisoner's Dilemma. Surprisingly, GPT-4o (a top-performing LLM across standard benchmarks) suffers the most substantial performance drop, suggesting that newer models are not addressing these issues. Interestingly, we found that a commonly used method of improving the reasoning capabilities of LLMs, chain-of-thought (CoT) prompting, reduces the biases in GPT-3.5, GPT-4o, and Llama-3-8B but increases the effect of the bias in GPT-4-Turbo, indicating that CoT alone cannot fully serve as a robust solution to this problem. We perform several additional experiments, which provide further insight into these observed behaviours.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
