忠实还是只是合理？评估闭源LLM在医学推理中的忠实性
Faithful or Just Plausible? Evaluating the Faithfulness of Closed-Source LLMs in Medical Reasoning
AI总结：
本文通过系统性黑盒评估，揭示闭源LLM在医学推理中的忠实性问题，发现链式推理步骤未必驱动预测，模型易受外部提示影响，强调忠实性比准确性更重要。
摘要：
闭源大型语言模型（LLM），如ChatGPT和Gemini，越来越多地被用于提供医疗建议，但其解释可能看似合理却未能反映模型的底层推理过程。此差距可能对患者和临床医生造成严重风险，因为他们可能信任连贯但误导性的解释。我们对三种广泛使用的闭源LLM进行了系统性的黑盒评估，以评估其在医学推理中的忠实性。我们的研究包含三种基于扰动的探针：（1）因果消融，测试陈述的链式思维（CoT）推理是否因果影响预测；（2）位置偏差，检查模型是否为由输入位置驱动的答案创建事后解释；（3）提示注入，测试模型对外部建议的易感性。我们还通过小规模的人类评估补充这些定量探针，以检查医生对解释忠实性的评估与普通人的信任感知之间的吻合程度。我们发现，CoT推理步骤往往并不因果驱动预测，且模型容易在不承认的情况下整合外部提示。相比之下，位置偏差在此设置中影响较小。这些结果强调，在评估用于医学的LLM时，忠实性而非准确性必须成为核心，以确保公众保护和安全的临床部署。
英文摘要：
Closed-source large language models (LLMs), such as ChatGPT and Gemini, are increasingly consulted for medical advice, yet their explanations may appear plausible while failing to reflect the model's underlying reasoning process. This gap poses serious risks as patients and clinicians may trust coherent but misleading explanations. We conduct a systematic black-box evaluation of faithfulness in medical reasoning among three widely used closed-source LLMs. Our study consists of three perturbation-based probes: (1) causal ablation, testing whether stated chain-of-thought (CoT) reasoning causally influences predictions; (2) positional bias, examining whether models create post-hoc justifications for answers driven by input positioning; and (3) hint injection, testing susceptibility to external suggestions. We complement these quantitative probes with a small-scale human evaluation of model responses to patient-style medical queries to examine concordance between physician assessments of explanation faithfulness and layperson perceptions of trustworthiness. We find that CoT reasoning steps often do not causally drive predictions, and models readily incorporate external hints without acknowledgment. In contrast, positional biases showed minimal impact in this setting. These results underscore that faithfulness, not just accuracy, must be central in evaluating LLMs for medicine, to ensure both public protection and safe clinical deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
