采用思维链(Chain-of-Thought, CoT)提示的大语言模型(Large Language Models, LLMs)拓宽了提升多步推理能力的研究范围。我们通常将多步推理分为两个阶段:路径生成阶段,用于生成推理路径;答案校准阶段,对推理路径进行后处理以得到最终答案。然而,现有文献缺乏对不同答案校准方法的系统性分析。本文总结了近期答案校准技术的分类体系,将其划分为步骤级和路径级两类策略。随后我们从统一视角对这些策略开展全面评估,系统考察了多路径下的步骤级与路径级答案校准方法。实验结果表明,融合两类策略的优势往往能得到最优结果。本研究有望为通过答案校准优化多步推理提供关键洞见。
英文摘要:
Large Language Models (LLMs) employing Chain-of-Thought (CoT) prompting have broadened the scope for improving multi-step reasoning capabilities. We generally divide multi-step reasoning into two phases: path generation to generate the reasoning path(s); and answer calibration post-processing the reasoning path(s) to obtain a final answer. However, the existing literature lacks systematic analysis on different answer calibration approaches. In this paper, we summarize the taxonomy of recent answer calibration techniques and break them down into step-level and path-level strategies. We then conduct a thorough evaluation on these strategies from a unified view, systematically scrutinizing step-level and path-level answer calibration across multiple paths. Experimental results reveal that integrating the dominance of both strategies tends to derive optimal outcomes. Our study holds the potential to illuminate key insights for optimizing multi-step reasoning with answer calibration.
Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models
Department of Computer Science and Engineering, Visvesvaraya Technological University, Belagavi(维斯瓦拉亚科技大学计算机科学与工程系,贝拉加维);Department of Computer Science and Business System, SG Balekundri Institute of Technology, Belagavi(SG巴莱昆德里理工学院计算机科学与商业系统系,贝拉加维)
Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, Beijing, China(中国科学院软件研究所信息处理实验室);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学);Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China(中国科学院大学网络安全学院);National Computer Network Emergency Response Technical Team/Coordination Center of China, Beijing, China(中国国家计算机网络应急技术配合中心)
Think Consistently, Reason Efficiently: Energy-Based Calibration for Implicit Chain-of-Thought
University of Oxford(牛津大学);Tsinghua University(清华大学);Tencent(腾讯);Southern University of Science and Technology(南方科技大学);National University of Singapore(新加坡国立大学)
Calibration-Aware Policy Optimization for Reasoning LLMs
National Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institution of Automation, Chinese Academy of Sciences(认知与复杂系统决策智能国家实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Beijing National Research Center for Information Science and Technology, Tsinghua University(北京信息科学与技术国家研究中心,清华大学)
Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning
Department of Electrical Engineering and Computer Science (MIT EECS)(麻省理工学院电气工程与计算机科学系);Computer Science and Artificial Intelligence Laboratory (MIT CSAIL)(麻省理工学院计算机科学与人工智能实验室);Laboratory for Information and Decision Systems (MIT LIDS)(麻省理工学院信息与决策系统实验室);Computational Science and Engineering (MIT CSE)(麻省理工学院计算科学与工程);Massachusetts Institute of Technology(麻省理工学院)
MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Models
University of Bristol(布里斯托大学);King Abdullah University of Science and Technology(卡利斯勒国王大学科学与技术学院);University of Georgia(佐治亚大学);Southern University of Science and Technology(南方科技大学)
Revisiting Uncertainty Estimation and Calibration of Large Language Models
School of Computer Science University of Sydney(悉尼大学计算机科学学院);City University of Hong Kong(香港城市大学);Shanghai Jiao Tong University(上海交通大学);Department of Engineering Science University of Oxford(牛津大学工程科学系)
Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
CLCG, University of Groningen(格罗宁根大学CLCG);University of Milano-Bicocca(米兰-布雷拉大学);University of Trieste(特里耶大学);Khoury College of Computer Sciences, Northeastern University(东北大学Khoury计算机科学学院)
Double-Calibration: Towards Reliable LLMs via Calibrating Knowledge and Reasoning Confidence
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China(中山大学计算机科学与工程学院,广州,中国);Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR(香港理工大学计算机系,香港特别行政区);School of Science and Technology, Hong Kong Metropolitan University, Hong Kong SAR(香港 Metropolitan 大学科技学院,香港特别行政区)
MediQAl: A French Medical Question Answering Dataset for Knowledge and Reasoning Evaluation
Data Clinic, University Hospital of Nantes, France(南特大学医院数据诊所,法国);Nantes Université, École Centrale Nantes, CNRS, LS2N, France(南特大学,中央理工学院南特分校,国家科学研究中心,LS2N,法国)
PAEC: Position-Aware Entropy Calibration for LLM Reasoning in RLVR
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);School of Computer Science and Technology, University of Chinese Academy of Sciences(中国科学院大学计算机科学与技术学院)