arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.21157cs.AIcs.CL

弥合算术鸿沟:认知复杂性基准与金融-PoT用于稳健的金融推理

Bridging the Arithmetic Gap: The Cognitive Complexity Benchmark and Financial-PoT for Robust Financial Reasoning

Tele-Communication Technology Bureau, Xinhua News Agency(新华社通信技术局)

Boxiang Zhao, Qince Li, Zhonghao Wang, Yi Wang, Peng Cheng, Bo Lin

AI总结:

本文提出认知复杂性基准和金融-PoT框架,通过架构解耦提升金融推理的鲁棒性,使Qwen3-235B模型在复杂任务中的准确率显著提高。

摘要:

尽管大型语言模型在语义任务上表现出色,但在金融定量推理方面却面临一个关键瓶颈,经常遭受“算术幻觉”和我们称为“认知崩溃”的系统性失效模式。为了严格量化这一现象,我们引入了认知复杂性基准(CCB),一个基于95份真实世界中国A股年度报告构建的数据集的稳健评估框架。与传统数据集不同,CCB将财务查询分为三维分类法,数据源、映射难度和结果单位,使在高认知负荷场景中推理退化能够被精确诊断。为了解决这些失败,我们提出了迭代双阶段金融-PoT框架。这种神经符号架构强制执行严格的架构解耦:它首先隔离语义变量提取和逻辑公式化,然后将计算卸载到一个迭代、自我纠正的Python沙盒中,以确保确定性执行。在CCB上的评估显示,尽管标准的思维链在复杂任务上表现不佳,我们的方法在稳健性上更优,将Qwen3-235B模型的平均准确率从59.7%提升到67.3%,并在高复杂性推理任务中实现了高达10倍的提升。这些发现表明,架构解耦是提高金融推理任务可靠性的重要因素,为需要语义理解和定量计算紧密对齐的高精度领域提供了可转移的架构洞察。

英文摘要:

While Large Language Models excel at semantic tasks, they face a critical bottleneck in financial quantitative reasoning, frequently suffering from "Arithmetic Hallucinations" and a systemic failure mode we term "Cognitive Collapse". To strictly quantify this phenomenon, we introduce the Cognitive Complexity Benchmark (CCB), a robust evaluation framework grounded in a dataset constructed from 95 real-world Chinese A-share annual reports. Unlike traditional datasets, the CCB stratifies financial queries into a three-dimensional taxonomy, Data Source, Mapping Difficulty, and Result Unit, enabling the precise diagnosis of reasoning degradation in high-cognitive-load scenarios. To address these failures, we propose the Iterative Dual-Phase Financial-PoT framework. This neuro-symbolic architecture enforces a strict architectural decoupling: it first isolates semantic variable extraction and logic formulation, then offloads computation to an iterative, self-correcting Python sandbox to ensure deterministic execution. Evaluation on the CCB demonstrates that while standard Chain-of-Thought falters on complex tasks, our approach offers superior robustness, elevating the Qwen3-235B model's average accuracy from 59.7\% to 67.3\% and achieving gains of up to 10-fold in high-complexity reasoning tasks. These findings suggest that architectural decoupling is a critical enabling factor for improving reliability in financial reasoning tasks, providing a transferable architectural insight for precision-critical domains that require tight alignment between semantic understanding and quantitative computation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑