评估大型语言模型作为实时战略代理：提供商性能、混合分解及时间风险游戏中的操作差距
Evaluating Large Language Models as Live Strategic Agents: Provider Performance, Hybrid Decomposition, and Operational Gaps in Timed Risk Play
AI总结：
本文研究了大型语言模型在实时策略环境中的表现，发现其性能受目标跟踪、执行转换、成本和运行时可靠性等因素影响，支持将LLM作为受限制工作流中的组件进行评估，而非孤立的基准测试对象。
摘要：
静态基准测试只能捕捉大型语言模型在实践中行为的一部分。实际系统将模型置于具有时间限制、格式约束和故障模式的重复循环中。我们研究了这种环境下的时间多阶段Risk游戏，其中包含明确的胜利目标和重复的规划与执行循环。在一项冻结规则的32局跨提供商锦标赛中，Gemini-3.1-Pro-Preview在32局中胜出20局，战胜了GPT-5.1、Claude-Opus-4-7和Kimi-K2.6。聚合的胜利分布与等强的空模型显著不同（p约1.5×10^-5）。随后，我们通过标准化执行在更便宜的Gemini Flash框架上进行分离。在该设计下，32局规划烘焙测试与近等值性一致（p约0.821），表明早期提供商差异主要来自端到端系统行为而非规划本身。为研究机制，我们分析了提供商锦标赛中保存的规划和执行轨迹。Gemini比其他模型更频繁地参考终端目标，且在胜利接近时增加这种关注。Gemini还更有效地将回合转化为深度征服链，尽管其运行时并不最干净。这些结果表明，实时代理性能取决于目标跟踪、执行转换、成本和运行时可靠性，并支持将LLM作为受限制工作流中的组件进行评估，而非孤立的基准测试响应者。
英文摘要：
Static benchmarks capture only part of how large language models behave in practice. Real systems place models inside repeated loops with time limits, formatting constraints, and failure modes. We study this setting in a timed multi-phase Risk environment with explicit victory targets and repeated planning and execution cycles. In a replicated 32-game cross-provider championship under frozen rules, gemini-3.1-pro-preview won 20 of 32 games against gpt-5.1, claude-opus-4-7, and kimi-k2.6, and the pooled winner distribution differs strongly from an equal-strength null (p approx 1.5 x 10^-5). We then separate planning from execution by standardizing execution on a cheaper Gemini Flash scaffold. Under this design, a pooled 32-game planner bakeoff is consistent with near-equality (p approx 0.821), which indicates that much of the earlier provider spread came from end-to-end system behavior rather than planning alone. To study mechanism, we analyze saved planning and execution traces from the provider championship. Gemini refers to the terminal objective far more often than the other models and increases that focus as victory approaches. Gemini also converts more turns into deep conquest chains, even though it is not the cleanest runtime. These results show that live-agent performance depends on objective tracking, execution conversion, cost, and runtime reliability, and they support evaluating LLMs as components in bounded workflows rather than as isolated benchmark respondents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
