arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.08134cs.CL
收藏

大型推理模型的置信度估计有多可靠?对高风险领域的系统基准测试

How Reliable are Confidence Estimators for Large Reasoning Models? A Systematic Benchmark on High-Stakes Domains

Michigan State University(密歇根州立大学);JPMorgan AI Research(摩根大通人工智能研究院);Henry Ford Health(亨利福特医疗集团)

Reza Khanmohammadi, Erfan Miahi, Simerjot Kaur, Ivan Brugere, Charese H. Smiley, Kundan Thind, Mohammad M. Ghassemi

AI总结:

本文通过系统基准测试,评估了大型推理模型置信度估计方法的可靠性,发现基于文本的编码器在歧视方面表现最佳,而结构感知模型在校准方面表现最佳,揭示了当前方法的局限性。

摘要:

大型推理模型(LRMs)的误校准会削弱其在高风险领域中的可靠性,需要方法来准确估计其长形式多步骤输出的置信度。为了解决这一缺口,我们引入了推理模型置信度估计基准(RMCB),这是一个包含来自六个流行LRMs的347,496个推理轨迹的公共资源,涵盖不同的架构家族。该基准从覆盖高风险领域的多样数据集构建,包括临床、金融、法律和数学推理,以及复杂的通用推理基准,所有样本均提供正确性注释。利用RMCB,我们对超过十种不同的基于表示的方法进行了大规模实证评估,涵盖顺序、基于图和基于文本的架构。我们的核心发现是歧视(AUROC)和校准(ECE)之间存在持续的权衡:基于文本的编码器在AUROC方面表现最佳(0.672),而结构感知模型在ECE方面表现最佳(0.148),没有单一方法在两者上都占优。此外,我们发现增加的架构复杂性并不总能优于更简单的顺序基线,这表明仅依赖片段级隐藏状态的方法存在性能上限。本工作提供了迄今为止最全面的该任务基准,建立了严格的基础线,并展示了当前基于表示的方法的局限性。

英文摘要:

The miscalibration of Large Reasoning Models (LRMs) undermines their reliability in high-stakes domains, necessitating methods to accurately estimate the confidence of their long-form, multi-step outputs. To address this gap, we introduce the Reasoning Model Confidence estimation Benchmark (RMCB), a public resource of 347,496 reasoning traces from six popular LRMs across different architectural families. The benchmark is constructed from a diverse suite of datasets spanning high-stakes domains, including clinical, financial, legal, and mathematical reasoning, alongside complex general reasoning benchmarks, with correctness annotations provided for all samples. Using RMCB, we conduct a large-scale empirical evaluation of over ten distinct representation-based methods, spanning sequential, graph-based, and text-based architectures. Our central finding is a persistent trade-off between discrimination (AUROC) and calibration (ECE): text-based encoders achieve the best AUROC (0.672), while structurally-aware models yield the best ECE (0.148), with no single method dominating both. Furthermore, we find that increased architectural complexity does not reliably outperform simpler sequential baselines, suggesting a performance ceiling for methods relying solely on chunk-level hidden states. This work provides the most comprehensive benchmark for this task to date, establishing rigorous baselines and demonstrating the limitations of current representation-based paradigms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑