arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.02230cs.AI
收藏

对财富管理流程的LLM代理进行基准测试

Benchmarking LLM Agents for Wealth-Management Workflows

Rory Milsom(独立研究者)

Rory Milsom

AI总结:

本研究通过构建12个任务对的基准,评估通用LLM代理在财富管理任务中的准确性和经济性,发现其性能受自主性水平和工作流程可靠性影响显著。

摘要:

现代工作依赖于多种数字协作工具,但常规流程仍然受到人为错误和延迟的困扰。为解决这一差距,本论文扩展了TheAgentCompany,引入了一个以金融为重点的环境,并探讨通用目的LLM代理是否能够准确且经济地完成具有代表性的财富管理任务。本研究引入了合成领域数据,丰富了同事模拟,并原型化了一个自动任务生成管道。本研究旨在创建并评估一个能够有意义地衡量代理在助理级财富管理工作中的适应性的评估集。我们构建了一个包含12个任务对的基准,涵盖检索、分析和综合/沟通,具有明确的接受标准和确定性评分者。我们引入了一组新的金融特定数据,并介绍了每个任务的高自主性和低自主性变体。论文得出结论,代理的限制更多在于数学推理,而非端到端工作流程的可靠性,并且受到自主性水平的显著影响,错误的模型评估已阻碍了基准测试。

英文摘要:

Modern work relies on an assortment of digital collaboration tools, yet routine processes continue to suffer from human error and delay. To address this gap, this dissertation extends TheAgentCompany with a finance-focused environment and investigates whether a general purpose LLM agent can complete representative wealth-management tasks both accurately and economically. This study introduces synthetic domain data, enriches colleague simulations, and prototypes an automatic task-generation pipeline. The study aims to create and assess an evaluation set that can meaningfully measure an agent's fitness for assistant-level wealth management work. We construct a benchmark of 12 task-pairs for wealth management assistants spanning retrieval, analysis, and synthesis/communication, with explicit acceptance criteria and deterministic graders. We seeded a set of new finance-specific data and introduced a high vs. low-autonomy variant of every task. The paper concluded that agents are limited less by mathematical reasoning and more so by end-to-end workflow reliability, and meaningfully affected by autonomy level, and that incorrect evaluation of models have hindered benchmarking.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑