评估大型语言模型中的创造力与欺骗能力：一个多智能体Balderdash的仿真框架
Evaluating Creativity and Deception in Large Language Models: A Simulation Framework for Multi-Agent Balderdash
AI总结：
本文提出一个基于Balderdash游戏的多智能体仿真框架，通过真实定义比率、欺骗比率和正确猜测比率等指标评估LLMs的创造力与逻辑推理能力，发现输入中的低频词汇会削弱其对规则与上下文的推理。
摘要：
大型语言模型（LLMs）在复杂任务和交互环境中展现出了令人印象深刻的能力，但其创造力仍未被充分探索。本文介绍了一个利用Balderdash游戏评估LLMs创造力和逻辑推理能力的仿真框架。在Balderdash中，玩家为生僻词汇编造虚构定义以欺骗他人，同时识别正确定义。我们的框架允许多个LLM智能体参与该游戏，评估其生成合理定义及基于游戏规则和历史进行策略制定的能力。我们实现了一个集中式游戏引擎，以各种LLMs作为参与者，并使用一个裁判LLM评估语义等价性。通过一系列实验，我们分析了不同LLMs的性能，考察了真实定义比率、欺骗比率和正确猜测比率等指标。结果揭示了LLMs的创造与欺骗能力，突出了其优势及待改进之处。具体而言，研究表明LLMs输入中的低频词汇会导致其对游戏规则和历史上下文的推理能力变差（https://github.com/ParsaHejabi/Simulation-Framework-for-Multi-Agent-Balderdash）。
英文摘要：
Large Language Models (LLMs) have shown impressive capabilities in complex tasks and interactive environments, yet their creativity remains underexplored. This paper introduces a simulation framework utilizing the game Balderdash to evaluate both the creativity and logical reasoning of LLMs. In Balderdash, players generate fictitious definitions for obscure terms to deceive others while identifying correct definitions. Our framework enables multiple LLM agents to participate in this game, assessing their ability to produce plausible definitions and strategize based on game rules and history. We implemented a centralized game engine featuring various LLMs as participants and a judge LLM to evaluate semantic equivalence. Through a series of experiments, we analyzed the performance of different LLMs, examining metrics such as True Definition Ratio, Deception Ratio, and Correct Guess Ratio. The results provide insights into the creative and deceptive capabilities of LLMs, highlighting their strengths and areas for improvement. Specifically, the study reveals that infrequent vocabulary in LLMs' input leads to poor reasoning on game rules and historical context (https://github.com/ParsaHejabi/Simulation-Framework-for-Multi-Agent-Balderdash).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
