更好的演示文稿还是不同的评审者？评估企业与投资银行中智能体框架的增益
Better Deck or Different Judge? Evaluating Agentic Harness Gains in Corporate and Investment Banking
浏览论文内容
中文总结 AI 辅助
本研究评估企业与投资银行中智能体框架的增益，通过LLM评审者评分发现系统生成演示文稿得分显著高于直接生成，但分数提升部分源于评审者变异性，而非纯粹文档质量改进。
中文摘要 AI 辅助
企业与投资银行团队使用演示文稿来支持信贷决策，并就融资和交易向客户提供建议。制作这些演示文稿需要核对财务数据、追溯来源并将分析转化为建议。我们回顾性地研究了一个结合27B语言模型、财务计算、叙述模板和验证检查的智能体框架的开发过程。LLM评审者指导工程变更并评估生成的演示文稿，这引发了一个问题：更高的分数是反映了更好的文档还是评分方式的变化。在去除模板标记的共享会话纯文本评分中，五位评审者对完整系统的评分比同一模型从简短提示直接生成的结果高出20.4至33.6分（满分95分）。每位评审者对所有十七个开发交付物均给予该系统更高的评分。与从简短提示直接生成Opus的差距范围为-4.7至+0.8分。评审者在开发轮次的总体进展上意见一致，但在最终演示文稿排名上的共识程度低于汇总分数。重复评分也会使未更改的演示文稿分数发生偏移，使得小幅改进难以与评审者变异性区分开来。
英文摘要
Corporate and investment banking teams use presentations to support credit decisions and advise clients on financing and transactions. Producing these decks requires reconciling financial data, tracing sources and turning analysis into a recommendation. We retrospectively study the development of an agentic harness combining a 27B language model, financial calculations, narrative templates and validation checks. LLM judges guide engineering changes and assess the resulting decks, raising the question of whether higher scores reflect better documents or changes in grading. In shared-session text-only grading with template markers removed, five judges score the complete system 20.4 to 33.6 points out of 95 above the same model generating directly from a short prompt. Every judge scores the system higher on all seventeen development deliverables. Margins against direct Opus generation from a short prompt range from -4.7 to +0.8 points. Judges agree on broad progress across development rounds but agree less on final-deck rankings than on pooled scores. Repeated grading also shifts scores on unchanged decks, making small improvements difficult to distinguish from judge variability.
发表机构
- AIData2Action
- TW3 Partners
机构由 AI 辅助整理，请以论文原文为准。
