SAG-Agent：通过动态知识图谱实现策略游戏中长周期推理
SAG-Agent: Enabling Long-Horizon Reasoning in Strategy Games via Dynamic Knowledge Graphs
AI总结：
SAG-Agent通过构建持久化的状态-动作图，解决无API环境下自主代理的效率瓶颈，提升探索效率和战略深度。
摘要：
大多数商用软件缺乏可访问的API，要求自主代理仅通过像素级图形用户界面进行交互。在无API的环境下，基于大语言模型（LLM）的代理面临严重效率瓶颈：受限于局部视觉体验，做出短视决策并依赖低效的试错，阻碍了技能获取和长周期规划。为克服这些限制，我们提出了SAG-Agent，一种经验驱动的学习框架，将代理的原始像素级交互结构化为持久的状态-动作图（SAG）。SAG-Agent通过拓扑链接功能相似但视觉不同的GUI状态，构建丰富的经验邻域，使代理能够从多样化的历史策略中进行泛化。为促进长周期推理，我们设计了一种基于图拓扑的新型混合内在奖励机制，结合状态价值奖励以利用已知的高价值路径和新颖性奖励以鼓励定向探索。这种方法将战略规划与纯粹发现解耦，使代理能够有效评估具有延迟回报的设置动作。我们在两个复杂的开放性GUI决策环境（文明六和slay the spire）中评估SAG-Agent，证明其在探索效率和战略深度方面显著优于现有方法。
英文摘要：
Most commodity software lacks accessible Application Programming Interfaces (APIs), requiring autonomous agents to interact solely through pixel-based Graphical User Interfaces (GUIs). In this API-free setting, large language model (LLM)-based agents face severe efficiency bottlenecks: limited to local visual experiences, they make myopic decisions and rely on inefficient trial-and-error, hindering both skill acquisition and long-horizon planning. To overcome these limitations, we propose SAG-Agent, an experience-driven learning framework that structures an agent's raw pixel-level interactions into a persistent State-Action Graph (SAG). SAG-Agent mitigates inefficient exploration by topologically linking functionally similar but visually distinct GUI states, constructing a rich neighborhood of experience that enables the agent to generalize from a diverse set of historical strategies. To facilitate long-horizon reasoning, we design a novel hybrid intrinsic reward mechanism based on the graph topology, combining a state-value reward for exploiting known high-value pathways with a novelty reward that encourages targeted exploration. This approach decouples strategic planning from pure discovery, allowing the agent to effectively value setup actions with delayed gratification. We evaluate SAG-Agent in two complex, open-ended GUI-based decision-making environments (Civilization V and Slay the Spire), demonstrating significant improvements in exploration efficiency and strategic depth over the state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
