arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.09101cs.CLcs.AIcs.IRcs.LG
收藏

面向多步推理中答案校准的统一视角

Towards A Unified View of Answer Calibration for Multi-Step Reasoning

National University of Singapore(新加坡国立大学);Zhejiang University(浙江大学)

Shumin Deng, Ningyu Zhang, Nay Oo, Bryan Hooi

AI总结:

针对多步推理中答案校准方法缺乏系统分析的问题,本文将答案校准技术分为步骤级与路径级两类,从统一视角开展全面评估,发现融合两类策略优势可获最优结果,为相关优化提供洞见。

摘要:

采用思维链(Chain-of-Thought, CoT)提示的大语言模型(Large Language Models, LLMs)拓宽了提升多步推理能力的研究范围。我们通常将多步推理分为两个阶段:路径生成阶段,用于生成推理路径;答案校准阶段,对推理路径进行后处理以得到最终答案。然而,现有文献缺乏对不同答案校准方法的系统性分析。本文总结了近期答案校准技术的分类体系,将其划分为步骤级和路径级两类策略。随后我们从统一视角对这些策略开展全面评估,系统考察了多路径下的步骤级与路径级答案校准方法。实验结果表明,融合两类策略的优势往往能得到最优结果。本研究有望为通过答案校准优化多步推理提供关键洞见。

英文摘要:

Large Language Models (LLMs) employing Chain-of-Thought (CoT) prompting have broadened the scope for improving multi-step reasoning capabilities. We generally divide multi-step reasoning into two phases: path generation to generate the reasoning path(s); and answer calibration post-processing the reasoning path(s) to obtain a final answer. However, the existing literature lacks systematic analysis on different answer calibration approaches. In this paper, we summarize the taxonomy of recent answer calibration techniques and break them down into step-level and path-level strategies. We then conduct a thorough evaluation on these strategies from a unified view, systematically scrutinizing step-level and path-level answer calibration across multiple paths. Experimental results reveal that integrating the dominance of both strategies tends to derive optimal outcomes. Our study holds the potential to illuminate key insights for optimizing multi-step reasoning with answer calibration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑