人类与机器：兵棋推演模拟中专家人类与语言模型的行为差异
Human vs. Machine: Behavioral Differences Between Expert Humans and Language Models in Wargame Simulations
AI总结：
本研究通过214名国家安全专家参与的兵棋推演实验，比较人类团队与LLM模拟团队在中美危机升级场景中的决策行为，发现LLM响应更具攻击性且受场景变化影响显著，在个体行动和战略倾向上与人类存在显著差异，提示政策制定者应谨慎对待AI自主决策和战略建议。
摘要：
对一些人而言，人工智能（AI）的出现有望改善决策并提高军事效能，同时减少人为错误和情绪的影响。然而，关于AI系统——尤其是可应用于众多任务的大语言模型（LLMs）——在高风险军事决策场景中与人类相比表现如何，仍存在争议，这类场景可能带来升级风险增大的潜在后果。为了检验这种潜在风险并审视将LLMs用于此类目的的做法，我们开展了一项新的兵棋推演实验，该实验有214名国家安全专家参与，旨在考察一个虚构的中美场景中的危机升级情况，并在独立的模拟中比较人类玩家团队的行为与LLM模拟的团队响应。在此，我们发现LLM模拟的响应可能更具攻击性，并且会显著受到场景变化的影响。我们展示了LLM与人类响应之间存在相当高水平的一致性，同时在个体行动和战略倾向上存在显著的定量和定性差异。这些差异取决于LLM在遵循战略指令时关于适当暴力水平的内在偏见、所选择的LLM，以及LLM是被要求直接为一个玩家团队做决策，还是先模拟一个玩家团队之间的对话。在模拟对话时，讨论缺乏质量，并维持着一种荒诞的和谐。LLM模拟无法体现人类玩家特征，即使对于“和平主义者”或“攻击性反社会者”等极端特质，也未显示出显著差异。在考察模拟中各个行动步骤之间的行为一致性时，所测试的LLM彼此之间存在偏差，但总体上表现出一定程度的一致行为。我们的研究结果促使政策制定者在授予自主权或遵循基于AI的战略建议之前保持谨慎。
英文摘要：
To some, the advent of artificial intelligence (AI) promises better decision-making and increased military effectiveness while reducing the influence of human error and emotions. However, there is still debate about how AI systems, especially large language models (LLMs) that can be applied to many tasks, behave compared to humans in high-stakes military decision-making scenarios with the potential for increased risks towards escalation. To test this potential and scrutinize the use of LLMs for such purposes, we use a new wargame experiment with 214 national security experts designed to examine crisis escalation in a fictional U.S.-China scenario and compare the behavior of human player teams to LLM-simulated team responses in separate simulations. Here, we find that the LLM-simulated responses can be more aggressive and significantly affected by changes in the scenario. We show a considerable high-level agreement in the LLM and human responses and significant quantitative and qualitative differences in individual actions and strategic tendencies. These differences depend on intrinsic biases in LLMs regarding the appropriate level of violence following strategic instructions, the choice of LLM, and whether the LLMs are tasked to decide for a team of players directly or first to simulate dialog between a team of players. When simulating the dialog, the discussions lack quality and maintain a farcical harmony. The LLM simulations cannot account for human player characteristics, showing no significant difference even for extreme traits, such as "pacifist" or "aggressive sociopath." When probing behavioral consistency across individual moves of the simulation, the tested LLMs deviated from each other but generally showed somewhat consistent behavior. Our results motivate policymakers to be cautious before granting autonomy or following AI-based strategy recommendations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
