arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.14857cs.AI
收藏

为《星际2》政策细化设计的世界模型

World Models for Policy Refinement in StarCraft II

The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences(认知与决策智能复杂系统重点实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学)

Yixin Zhang, Ziyi Wang, Yiming Rong, Haoxi Wang, Jinling Jiang, Shuang Xu, Haoran Wu, Shiyu Zhou, Bo Xu

AI总结:

StarWM为《星际2》政策细化设计的世界模型,通过预测未来观察和结构化文本表示提升策略决策性能。

摘要:

大型语言模型(LLMs)近期展示了强大的推理和泛化能力,推动其作为复杂环境中的决策政策使用。《星际2》(SC2)因其巨大的状态-动作空间和部分可观测性,是一个具有挑战性的测试平台。然而,现有的基于LLM的SC2智能体主要集中在改进自身策略,而忽略了将可学习的、动作条件的转移模型整合到决策循环中。为弥合这一差距,我们提出了StarWM,首个为SC2设计的世界模型,能够预测部分可观测性下的未来观察。为了促进学习SC2的混合动态,我们引入了一种结构化的文本表示,将观察分解为五个语义模块,并构建了SC2-Dynamics-50k,首个针对SC2动态预测的指令微调数据集。我们进一步开发了多维离线评估框架,用于评估预测的结构化观察。离线结果表明,StarWM在零样本基线上的显著提升,包括资源预测准确率近60%的改进以及自我方宏观情况一致性。最后,我们提出了StarWM-Agent,一个整合StarWM的世界模型增强决策系统,通过生成-模拟-细化的决策循环实现前瞻性政策细化。在线评估与SC2内置AI的对比显示,一致的改进,分别在Hard(LV5)、Harder(LV6)和VeryHard(LV7)中获得30%、15%和30%的胜率提升,同时改善了宏观管理稳定性和战术风险评估。

英文摘要:

Large Language Models (LLMs) have recently shown strong reasoning capabilities, motivating their use in complex decision-making environments. StarCraft II (SC2), with its massive state-action space and partial observability, is a challenging testbed. However, existing LLM-based SC2 agents primarily focus on improving the policy itself, leaving the integration of a learnable, action-conditioned dynamics model into the decision loop largely unexplored. In this work, we propose StarWM and StarWM-Agent, and conduct the first systematic study of the learnability and decision utility of player-view, action-conditioned textual world models for SC2. StarWM predicts short-horizon future observations under partial observability. StarWM-Agent integrates StarWM into a lightweight Generate-Simulate-Refine loop for foresight-driven policy refinement. Extensive experiments show that StarWM substantially outperforms zero-shot baselines across multiple dimensions, while StarWM-Agent achieves consistent win-rate gains of 30%, 15%, and 30% against the SC2 built-in AI at Hard (LV5), Harder (LV6), and VeryHard (LV7), respectively. Additional analyses show that our method is complementary to existing history-summarization SC2-agent approaches, with their combination achieving the strongest online performance in our evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑