LLMs能够通过教学式思维链发现数学推理错误
LLMs can Find Mathematical Reasoning Mistakes by Pedagogical Chain-of-Thought
AI总结：
本文提出受布鲁姆认知模型启发的教学式思维链（PedCoT）提示策略，通过两阶段交互和教学原则设计，显著提升LLMs在零样本条件下识别数学推理错误的能力，为自动评分奠定基础。
摘要：
自我修正正逐渐成为缓解大语言模型（LLMs）幻觉问题的一种有前景的方法。为了促进有效的自我修正，近期研究提出将错误检测作为其初始步骤。然而，现有文献表明，当使用过于简单的提示策略时，LLMs往往难以可靠地识别推理错误。为应对这一挑战，我们引入了一种独特的提示策略，称为教学式思维链（PedCoT），该策略专门用于指导推理错误的识别，尤其是数学推理错误。PedCoT由提示设计教学原则（PPP）、两阶段交互过程（TIP）和基于依据的PedCoT提示组成，这些均受到布鲁姆认知模型（BCM）教育理论的启发。我们在两个包含不同难度数学题的公开数据集上评估了我们的方法。实验表明，我们的零样本提示策略显著优于强基线。所提出的方法能够实现可靠的数学错误识别目标，并为自动数学答案评分奠定基础。结果强调了教育理论作为领域知识，在指导提示策略设计以有效应对LLMs挑战性任务中的重要意义。
英文摘要：
Self-correction is emerging as a promising approach to mitigate the issue of hallucination in Large Language Models (LLMs). To facilitate effective self-correction, recent research has proposed mistake detection as its initial step. However, current literature suggests that LLMs often struggle with reliably identifying reasoning mistakes when using simplistic prompting strategies. To address this challenge, we introduce a unique prompting strategy, termed the Pedagogical Chain-of-Thought (PedCoT), which is specifically designed to guide the identification of reasoning mistakes, particularly mathematical reasoning mistakes. PedCoT consists of pedagogical principles for prompts (PPP) design, two-stage interaction process (TIP) and grounded PedCoT prompts, all inspired by the educational theory of the Bloom Cognitive Model (BCM). We evaluate our approach on two public datasets featuring math problems of varying difficulty levels. The experiments demonstrate that our zero-shot prompting strategy significantly outperforms strong baselines. The proposed method can achieve the goal of reliable mathematical mistake identification and provide a foundation for automatic math answer grading. The results underscore the significance of educational theory, serving as domain knowledge, in guiding prompting strategy design for addressing challenging tasks with LLMs effectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
