FinToolBench：评估LLM代理在真实金融工具使用中的表现
FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Use
AI总结：
FinToolBench是首个评估金融工具学习代理真实世界表现的基准，通过760个可执行金融工具和295个严格查询，评估时效性、意图类型和合规性等关键维度。
摘要：
大型语言模型（LLMs）在金融领域的整合正在推动从被动信息检索向动态、代理式交互的转变。尽管通用工具学习已见证基准的激增，但金融领域，以高风险、严格合规和快速数据波动为特征，仍然严重不足。现有的金融评估主要集中在静态文本分析或基于文档的问答，忽略了工具执行的复杂现实。相反，通用工具基准缺乏金融所需的领域特定严谨性，往往依赖于玩具环境或少量金融API。为弥合这一差距，我们引入FinToolBench，首个专注于评估金融工具学习代理的真实世界可运行基准。与以往仅限于少量模拟工具不同，FinToolBench建立了一个现实的生态系统，将760个可执行的金融工具与295个严格要求工具的查询相结合。我们提出了一种新的评估框架，超越二元执行成功，评估代理在金融关键维度：时效性、意图类型和监管领域一致性。此外，我们提出了FATR，一个金融意识的工具检索和推理基线，增强了稳定性和合规性。通过提供首个可审计、代理式金融执行的测试床，FinToolBench为金融领域的可信AI设定了新标准。工具清单、执行环境和评估代码将开源以促进未来研究。
英文摘要：
The integration of Large Language Models (LLMs) into the financial domain is driving a paradigm shift from passive information retrieval to dynamic, agentic interaction. While general-purpose tool learning has witnessed a surge in benchmarks, the financial sector, characterized by high stakes, strict compliance, and rapid data volatility, remains critically underserved. Existing financial evaluations predominantly focus on static textual analysis or document-based QA, ignoring the complex reality of tool execution. Conversely, general tool benchmarks lack the domain-specific rigor required for finance, often relying on toy environments or a negligible number of financial APIs. To bridge this gap, we introduce FinToolBench, the first real-world, runnable benchmark dedicated to evaluating financial tool learning agents. Unlike prior works limited to a handful of mock tools, FinToolBench establishes a realistic ecosystem coupling 760 executable financial tools with 295 rigorous, tool-required queries. We propose a novel evaluation framework that goes beyond binary execution success, assessing agents on finance-critical dimensions: timeliness, intent type, and regulatory domain alignment. Furthermore, we present FATR, a finance-aware tool retrieval and reasoning baseline that enhances stability and compliance. By providing the first testbed for auditable, agentic financial execution, FinToolBench sets a new standard for trustworthy AI in finance. The tool manifest, execution environment, and evaluation code will be open-sourced to facilitate future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
