EcoGym：评估基于LLM的长周期计划与执行能力于交互经济中
EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies
AI总结：
EcoGym提出一个通用基准，用于评估LLM在交互经济中的长周期计划与执行能力，通过三个环境展示不同策略和行动的优化挑战。
摘要：
长周期规划被认为是自主LLM代理的核心能力，但现有评估框架多为 episodic、领域特定或缺乏对持久经济动态的建模。本文引入EcoGym，一个通用基准，用于连续计划与执行决策的交互经济。EcoGym包含三个环境：Vending（改编自闭源的Vending-Bench，完全开源）、Freelance（新）和Operation（新），通过统一的决策过程和标准化接口实现。评估基于商业相关结果（如净资产、收入和DAU），旨在评估长期战略一致性和在部分可观测性和随机性下的鲁棒性。实验显示，不同LLM在三种场景中均存在系统性矛盾，无单一模型在所有场景中占优。EcoGym作为开放、可扩展的测试平台，用于透明的长周期代理评估和研究经济场景中的可控性效用权衡。
英文摘要：
Long-horizon planning is widely recognized as a core capability of autonomous LLM-based agents; however, current evaluation frameworks suffer from being largely episodic, domain-specific, or insufficiently grounded in persistent economic dynamics. We introduce EcoGym, a generalizable benchmark for continuous plan-and-execute decision making in interactive economies. EcoGym comprises three diverse environments: Vending (adapted from the closed-source Vending-Bench, with full open-source release), Freelance (new), and Operation (new), implemented in a unified decision-making process with standardized interfaces, and budgeted actions over an effectively unbounded horizon (1000+ steps if 365 day-loops for evaluation). The evaluation of EcoGym is based on business-relevant outcomes (e.g., net worth, income, and DAU), targeting long-term strategic coherence and robustness under partial observability and stochasticity. Experiments across eleven leading LLMs expose a systematic tension: no single model dominates across all three scenarios. Critically, we find that models exhibit significant suboptimality in either high-level strategies or efficient actions executions. EcoGym is released as an open, extensible testbed for transparent long-horizon agent evaluation and for studying controllability utility trade-offs in economic settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
