大语言模型在医疗诊断中的可靠性：一致性、可操纵性与情境感知能力的检验
The Reliability of LLMs for Medical Diagnosis: An Examination of Consistency, Manipulation, and Contextual Awareness
AI总结：
该研究检验Google Gemini 2.0 Flash与OpenAI ChatGPT-4o的医疗诊断可靠性，发现二者虽一致性达100%，但易受无关信息操纵，且情境响应存在差异，提示其医疗应用需临床监督与结构化保障。
摘要：
本研究从三个维度评估了两款大语言模型（LLMs）——Google Gemini 2.0 Flash和OpenAI ChatGPT-4o的诊断可靠性：重新表述输入下的一致性、对无关提示内容的敏感性、对新增临床情境的响应性。我们设计了52个临床场景，并在受控条件下对每个场景进行修改：一致性检验中，对场景进行人口统计学、措辞、检查方式的重新表述，保留诊断核心；敏感性评估中，嵌入无关但合理的叙事细节，同时保持临床证据不变；情境感知检验中，加入患者病史、生活方式数据或诊断结果，以改变预期诊断。随后由医师评审判断情境驱动的改变是否符合临床规范。两款模型在所有等效变体及重复查询中均给出相同诊断，一致性达100%；加入无关细节后，Gemini在40.0%的案例中改变诊断，ChatGPT则为30.0%。ChatGPT对情境的响应频率高于Gemini（77.8% vs. 55.6%），但其中临床不恰当改变的占比更大（33.3% vs. 22.2%）；Gemini由情境驱动的改变更常被判定为恰当（66.7% vs. 55.6%）。受控输入下的一致性无法保护两款模型免受无关操纵或情境变化导致的不合理诊断偏移。在LLMs能够区分相关与无关输入并标记证据不足前，其诊断应用需临床医师监督及结构化保障措施。
英文摘要：
This study evaluated the diagnostic reliability of two Large Language Models (LLMs), Google Gemini 2.0 Flash and OpenAI ChatGPT-4o, across three dimensions: consistency under rephrased inputs, susceptibility to irrelevant prompt content, and responsiveness to added clinical context. We designed 52 clinical scenarios and modified each under controlled conditions. For consistency, scenarios were rephrased with demographic, wording, and examination changes that preserved the diagnostic core. And the susceptibility was evaluated through embedding irrelevant but plausible narrative details while keeping the clinical evidence unchanged. For contextual awareness, patient history, lifestyle data, or diagnostic findings were added to shift the expected diagnosis. Physician reviewers then judged whether context-driven changes were clinically appropriate. Both models returned identical diagnoses across all equivalent variants and repeated queries (100% consistency). When irrelevant details were added, Gemini changed its diagnosis in 40.0% of cases and ChatGPT in 30.0%. ChatGPT responded to context more often than Gemini (77.8% vs. 55.6%), but a larger share of its changes were clinically inappropriate (33.3% vs. 22.2%). Gemini's context-driven changes were more often judged appropriate (66.7% vs. 55.6%). Consistency under controlled inputs did not protect either model from irrelevant manipulation or unjustified diagnostic shifts when context changed. Before LLMs can separate relevant from irrelevant input and flag insufficient evidence, their diagnostic use requires clinician oversight and structured safeguards.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
