arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.13693cs.CYcs.AI
收藏

ESG叙述评分中重度推理LLM部署的有限边际收益:一项关于日本上市公司的4模型共识研究

Limited Marginal Benefit of Reasoning-Heavy LLM Deployment in ESG Narrative Scoring: A 4-Model Consensus Study on Japanese Listed Firms

Kansai University(关西大学)

Hiroyuki Kokubu

AI总结:

通过4模型共识设计,研究在ESG叙述评分中,重度推理模型相比非推理模型是否带来显著收益,发现其边际收益有限且成本高昂。

摘要:

使用大语言模型(LLM)对ESG叙述披露进行自动评分正逐渐受到关注,但重度推理的前沿模型是否带来与其成本相称的价值,在实证上仍不确定。我们基于十家日本上市公司的语料库,沿三个评分轴——定量目标、进度跟踪基础设施和外部标准对齐——通过四模型共识设计评估这一问题,该设计将一个推理型前沿模型与三个非推理型同期模型相结合。在120个公司×轴×模型评分中,推理型模型与每个非推理型模型之间的汇总平均绝对偏差为0.38(5分制);仅有2%的成对比较达到两分偏差,且无任何比较超过两分。每公司成本核算显示,仅推理型模型的成本约为三个非推理型模型集成成本的5.6倍,而结果仅在小范围内存在差异。我们得出结论,在基于区间的ESG叙述评分中,重度推理部署相对于非推理共识并未显著改善结果,同时大幅增加了运营成本。我们讨论了这对成本效益的ESG自动评分流程以及应用问责环境中的LLM部署治理的启示。本工作的早期版本可在SSRN上获取(摘要ID 6683303)。

英文摘要:

Automated scoring of ESG narrative disclosures with large language models (LLMs) is gaining traction, yet whether reasoning-heavy frontier models add value commensurate with their cost remains empirically unsettled. We evaluate this question on a corpus of ten Japanese listed firms across three rubric axes -- quantitative targets, progress-tracking infrastructure, and external-standard alignment -- using a four-model consensus design that combines a reasoning-on frontier model with three reasoning-off contemporaries. Across 120 firm x axis x model scores, the pooled mean absolute deviation between the reasoning-on model and each reasoning-off counterpart is 0.38 on a 5-point scale; only 2% of pairwise comparisons reach a two-point deviation, and none exceeds two points. Per-firm cost accounting shows the reasoning-on arm alone costs roughly 5.6x as much as the three-provider reasoning-off ensemble, for outcomes that differ only within small margins. We conclude that in span-based ESG narrative scoring, reasoning-heavy deployment does not materially improve outcomes relative to reasoning-off consensus, while substantially increasing operational cost. We discuss implications for cost-effective ESG auto-scoring pipelines and LLM deployment governance in applied accountability settings. An earlier version of this work is available on SSRN (Abstract ID 6683303).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑