LLM代理能成为CFO吗?在不确定的企业环境中评估长期资源分配
Can LLM Agents Be CFOs? Benchmarking Long-Horizon Resource Allocation in an Uncertain Enterprise Environment
AI总结:
本文通过EnterpriseArena模拟器评估LLM在不确定环境下的长期资源分配能力,发现现有模型在复杂任务中表现不足,仅15.4%的试验能持续完整周期。
摘要:
大型语言模型(LLM)代理在复杂任务上的测试日益增加,但其在长期资源分配中的能力仍不明确。本文引入EnterpriseArena,一个132个月的CFO模拟器,用于评估在金融科技贷款公司中不确定环境下的长期资源分配。代理必须管理流动性、结账、收集成本信号并请求股权或债务融资,以应对不断变化的宏观经济环境。模拟器基于转换后的公司财务数据、匿名商业文档、十年期宏观经济和行业信号以及专家验证的操作规则构建。在23个LLM和四个代理框架的实验中发现,当前代理仍远未稳健:仅15.4%的试验能持续完整周期,较大模型并不总能优于较小模型,且失败会跨观测、行动时间和资本规模传播。这些发现确立了在不确定环境下长期资源分配作为LLM代理的一个独特能力缺口。
英文摘要:
Large language model (LLM) agents are increasingly tested on complex tasks, but their ability to allocate scarce resources over long horizons remains unclear. Unlike reactive tasks with immediate feedback, this setting requires agents to make binding commitments under partial observability, delayed consequences, hard resource budgets, and shifting dynamics. We introduce EnterpriseArena, a 132-month CFO simulator that evaluates long-horizon resource allocation under uncertainty in a FinTech lending firm. Agents must manage liquidity, close books, gather costly signals, and request equity or debt financing across changing macroeconomic regimes. The simulator is built from transformed firm-level financial data, anonymized business documents, decade-scale macroeconomic and industry signals, and expert-validated operating rules. Experiments across 23 LLMs and four agent frameworks show that current agents remain far from robust: only 15.4% of trials survive the full horizon, larger models do not reliably outperform smaller ones, and failures cascade across observation, action timing, and capital sizing. These findings establish long-horizon resource allocation under uncertainty as a distinct capability gap for LLM agents.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。