arXiv 2604.002612026-04-02cs.CL

大语言模型能否在医疗问答中自我纠正？一项探索性研究

Can Large Language Models Self-Correct in Medical Question Answering? An Exploratory Study

Department of Electrical and Computer Engineering, University of Minnesota（明尼苏达大学电气与计算机工程系）；Division of Computational Health Sciences, Department of Surgery, University of Minnesota（明尼苏达大学外科学系计算健康科学部）；Department of Software Engineering, University of St. Thomas（圣托马斯大学软件工程系）

Zaifu Zhan, Mengyuan Cui, Rui Zhang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文探讨大语言模型在医疗问答中的自我反思能力，通过比较标准Co-T提示与迭代自我反思循环，发现自我反思并不总能提升准确性，且效果依赖于数据集和模型。

摘要：

大语言模型（LLMs）在医疗问答（medical QA）中表现出色，链式思维（CoT）提示通过引导显式中间推理进一步提升了结果；同时，自我反思（自我纠正）提示被广泛声称能通过促使LLMs批判和修订自身推理来增强模型可靠性，但在安全关键的医疗环境中其有效性仍不明确。本文对自我反思推理在医疗多选问答中的探索性分析：使用GPT-4o和GPT-4o-mini，比较标准CoT提示与迭代自我反思循环，并跟踪在三个广泛使用的医疗QA基准（MedQA、HeadQA和PubMedQA）上预测如何随反思步骤演变。我们分析自我反思是否导致错误纠正、错误持续或引入新错误。结果表明，自我反思提示并不总能提升准确性，其影响高度依赖于数据集和模型：在MedQA上产生适度提升，但在HeadQA和PubMedQA上提供有限或负效益，增加反思步骤数量并不保证更好性能。这些发现凸显了推理透明度与推理正确性之间的差距，表明自我反思推理应被视为理解模型行为的分析工具，而非提升医疗QA可靠性的独立解决方案。

英文摘要：

Large language models (LLMs) have achieved strong performance on medical question answering (medical QA), and chain-of-thought (CoT) prompting has further improved results by eliciting explicit intermediate reasoning; meanwhile, self-reflective (self-corrective) prompting has been widely claimed to enhance model reliability by prompting LLMs to critique and revise their own reasoning, yet its effectiveness in safety-critical medical settings remains unclear. In this work, we conduct an exploratory analysis of self-reflective reasoning for medical multiple-choice question answering: using GPT-4o and GPT-4o-mini, we compare standard CoT prompting with an iterative self-reflection loop and track how predictions evolve across reflection steps on three widely used medical QA benchmarks (MedQA, HeadQA, and PubMedQA). We analyze whether self-reflection leads to error correction, error persistence, or the introduction of new errors. Our results show that self-reflective prompting does not consistently improve accuracy and its impact is highly dataset- and model-dependent: it yields modest gains on MedQA but provides limited or negative benefits on HeadQA and PubMedQA, and increasing the number of reflection steps does not guarantee better performance. These findings highlight a gap between reasoning transparency and reasoning correctness, suggesting that self-reflective reasoning is better viewed as an analytical tool for understanding model behavior rather than a standalone solution for improving medical QA reliability.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。