提示语言影响大型语言模型的诊断推理和准确性
Prompting language influences diagnostic reasoning and accuracy of large language models
AI总结：
本研究探讨了提示语言对大型语言模型在临床诊断推理和准确性上的影响，通过比较英语和法语性能，发现四种模型在英语环境下表现更优，而o3模型未表现出语言效应，表明提示语言是影响模型临床性能的关键因素。
摘要：
大型语言模型（LLMs）越来越多地被探索用于临床决策支持，但大多数评估都是用英语进行的，这使得其在其他语言中的可靠性存疑。本文通过比较五种LLM（o3、DeepSeek-R1、GPT-4-Turbo、Llama-3.1-405B-Instruct和BioMistral-7B）在英语和法语环境下的表现，评估了提示语言对诊断推理和最终诊断准确性的影响。总共评估了180个涵盖16个医学专科的临床情景，由两位医生使用18分量表评估诊断准确性和推理质量。五种模型中有四种在英语环境下表现更优（平均差异0.37-0.91，调整p<0.05），差距涵盖多个推理方面，包括鉴别诊断、逻辑结构和内部效度。o3是唯一一个未表现出整体语言效应的模型。这些发现表明，提示语言仍然是影响LLM临床性能的关键因素，对全球公平的语义文化部署具有重要影响。
英文摘要：
Large language models (LLMs) are increasingly explored for clinical decision support, yet most evaluations are conducted in English, leaving their reliability in other languages uncertain. Here we evaluate the impact of prompting language on diagnostic reasoning and final diagnosis accuracy by comparing English and French performance across five LLMs (o3, DeepSeek-R1, GPT-4-Turbo, Llama-3.1-405B-Instruct, and BioMistral-7B). A total of 180 clinical vignettes covering 16 medical specialties were assessed by two physicians using an 18-point scale evaluating both diagnosis accuracy and reasoning quality. Four of the five models performed better in English (mean difference 0.37-0.91, adjusted p < 0.05), with the gap spanning multiple aspects of reasoning, including differential diagnosis, logical structure, and internal validity. o3 was the only model showing no overall language effect. These findings demonstrate that prompting language remains a critical determinant of LLM clinical performance, with implications for equitable linguistico-cultural deployment worldwide.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
