推理模型近期在数学和编程等领域展现出显著进展。然而，它们在数学和编程方面的专家级能力，与其在诸如网页导航和电脑/手机使用等长周期交互任务中的表现形成了鲜明对比。受人类认知相关文献的启发，我们认为当前的AI智能体需要“替代性试错”——即在行动前在心理上模拟不同未来的能力——以增强其在复杂交互环境中的理解力和表现。我们提出了Dyna-Mind，这是一个两阶段训练框架，明确教导(V)LM智能体将此类模拟整合进其推理中。在阶段1，我们提出了Reasoning with Simulations (ReSim)，它训练智能体从扩展搜索树中生成结构化推理轨迹，这些搜索树基于通过环境交互收集的真实经验构建。因此，ReSim将智能体的推理建立在可靠的世界动态之上，并赋予其在推理中预测未来状态的能力。在阶段2，我们提出了Dyna-GRPO，这是一种在线强化学习方法，通过将结果奖励和中间状态作为真实rollout的反馈，进一步增强智能体的模拟与决策能力。在两个合成基准（Sokoban和ALFWorld）及一个真实基准（AndroidWorld）上的实验表明：(1) ReSim有效地将模拟能力注入到AI智能体中；(2) Dyna-GRPO利用结果和交互级别的信号，为长周期、重规划任务学习到了更好的策略。这些结果共同突显了模拟在使AI智能体于日益具挑战性的环境中进行推理、规划和行动方面所发挥的核心作用。
英文摘要：
Reasoning models have recently shown remarkable progress in domains such as math and coding. However, their expert-level abilities in math and coding contrast sharply with their performance in long-horizon, interactive tasks such as web navigation and computer/phone-use. Inspired by literature on human cognition, we argue that current AI agents need ''vicarious trial and error'' - the capacity to mentally simulate alternative futures before acting - in order to enhance their understanding and performance in complex interactive environments. We introduce Dyna-Mind, a two-stage training framework that explicitly teaches (V)LM agents to integrate such simulation into their reasoning. In stage 1, we introduce Reasoning with Simulations (ReSim), which trains the agent to generate structured reasoning traces from expanded search trees built from real experience gathered through environment interactions. ReSim thus grounds the agent's reasoning in faithful world dynamics and equips it with the ability to anticipate future states in its reasoning. In stage 2, we propose Dyna-GRPO, an online reinforcement learning method to further strengthen the agent's simulation and decision-making ability by using both outcome rewards and intermediate states as feedback from real rollouts. Experiments on two synthetic benchmarks (Sokoban and ALFWorld) and one realistic benchmark (AndroidWorld) demonstrate that (1) ReSim effectively infuses simulation ability into AI agents, and (2) Dyna-GRPO leverages outcome and interaction-level signals to learn better policies for long-horizon, planning-intensive tasks. Together, these results highlight the central role of simulation in enabling AI agents to reason, plan, and act more effectively in the ever more challenging environments.
Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning
Hebrew University of Jerusalem（海法大学）；MPI for Biological Cybernetics（生物感知研究所）；Centre for AI and Machine Learning, AU（人工智能与机器学习中心）；IoPPN, King’s College London（国王学院伦敦人工智能与感知神经科学中心）
From Representation to Enactment: The ABC Framework of the Translating Mind
Center for Research and Innovation in Translation and Translation Technology, Kent State University（肯特州立大学翻译与翻译技术研究与创新中心）；Rikkyo University（立教大学）；City University of Hong Kong（香港城市大学）
Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Task
Stanford University（斯坦福大学）；Australian National University（澳大利亚国立大学）；University of St. Andrews（圣安德鲁斯大学）；University of Cambridge（剑桥大学）；University of California, San Diego（加利福尼亚大学圣地亚哥分校）
