精通《达芬奇密码》：基于Transformer、LLM和PPO的智能体比较研究
Mastering Da Vinci Code: A Comparative Study of Transformer, LLM, and PPO-based Agents
AI总结：
针对《达芬奇密码》不完全信息逻辑推理博弈的挑战，该研究对比了Transformer基线、结构化提示引导的LLM及带Transformer编码器的PPO三类智能体，发现PPO智能体胜率达58.5%±1.0%最优，揭示了深度强化学习在复杂推理任务中的优势及现有LLM的局限。
摘要：
《达芬奇密码》是一款逻辑推理类不完全信息博弈，对人工智能提出了独特挑战，要求AI具备超越简单模式识别的精细推理能力。本文探究了多种AI范式在精通该游戏方面的效能。我们开发并评估了三种不同的智能体架构：仅具备有限历史上下文的基于Transformer的基线模型、由结构化提示引导的多款大语言模型（LLM）智能体（包括Gemini、DeepSeek和GPT系列变体），以及采用Transformer编码器处理完整游戏历史的基于近端策略优化（PPO）的智能体。我们以基线模型为基准进行性能测试，其中基于PPO的智能体展现出最优胜率（$58.5\\% \pm 1.0\\%$），显著优于各类LLM智能体。我们的分析凸显了深度强化学习在复杂推理任务策略优化方面的优势，尤其是在自博弈过程中学习隐性策略的能力。我们还考察了当前LLM尽管有复杂提示工程加持，在长时间游戏中保持严格逻辑一致性和策略深度方面的能力与固有局限。本研究有助于更广泛地理解AI在涉及隐藏信息和多步逻辑推理的休闲游戏中的应用，为有效智能体设计及不同AI方法的比较优势提供了洞见。
英文摘要：
The Da Vinci Code, a game of logical deduction and imperfect information, presents unique challenges for artificial intelligence, demanding nuanced reasoning beyond simple pattern recognition. This paper investigates the efficacy of various AI paradigms in mastering this game. We develop and evaluate three distinct agent architectures: a Transformer-based baseline model with limited historical context, several Large Language Model (LLM) agents (including Gemini, DeepSeek, and GPT variants) guided by structured prompts, and an agent based on Proximal Policy Optimization (PPO) employing a Transformer encoder for comprehensive game history processing. Performance is benchmarked against the baseline, with the PPO-based agent demonstrating superior win rates ($58.5\% \pm 1.0\%$), significantly outperforming the LLM counterparts. Our analysis highlights the strengths of deep reinforcement learning in policy refinement for complex deductive tasks, particularly in learning implicit strategies from self-play. We also examine the capabilities and inherent limitations of current LLMs in maintaining strict logical consistency and strategic depth over extended gameplay, despite sophisticated prompting. This study contributes to the broader understanding of AI in recreational games involving hidden information and multi-step logical reasoning, offering insights into effective agent design and the comparative advantages of different AI approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
