结合多个大型语言模型的见解可提高诊断准确性
Combining Insights From Multiple Large Language Models Improves Diagnostic Accuracy
AI总结：
本研究通过聚合多个大型语言模型的响应生成鉴别诊断，在200个真实病例数据集上验证其准确率显著优于单个模型，为提升LLMs诊断可靠性并摆脱单一供应商依赖提供了集体智能方案。
摘要：
背景：诸如OpenAI的GPT-4或Google的PaLM 2等大型语言模型（LLMs）被提议作为可行的诊断支持工具，甚至被认为可以替代“路边咨询”。然而，即使是专门针对医学主题训练的LLMs，在实际应用中也可能缺乏足够的诊断准确性。方法：利用集体智能方法和一个包含200个真实病例临床小插曲的数据集，我们评估并比较了通过询问单个商业LLMs（OpenAI GPT-4、Google PaLM 2、Cohere Command、Meta Llama 2）获得的鉴别诊断的准确性，与通过聚合相同LLMs组合的响应所合成的鉴别诊断的准确性。结果：我们发现，聚合来自多个不同LLMs的响应可产生更准确的鉴别诊断（3个LLMs的平均准确率：75.3%±1.6个百分点），相比之下，单个LLMs产生的鉴别诊断（单个LLMs的平均准确率：59.0%±6.1个百分点）。讨论：利用集体智能方法合成结合不同LLMs响应的鉴别诊断，实现了推动LLMs作为诊断支持工具被接受所需的两个必要步骤：（1）展示高诊断准确性，以及（2）消除对单一商业供应商的依赖。
英文摘要：
Background: Large language models (LLMs) such as OpenAI's GPT-4 or Google's PaLM 2 are proposed as viable diagnostic support tools or even spoken of as replacements for "curbside consults". However, even LLMs specifically trained on medical topics may lack sufficient diagnostic accuracy for real-life applications. Methods: Using collective intelligence methods and a dataset of 200 clinical vignettes of real-life cases, we assessed and compared the accuracy of differential diagnoses obtained by asking individual commercial LLMs (OpenAI GPT-4, Google PaLM 2, Cohere Command, Meta Llama 2) against the accuracy of differential diagnoses synthesized by aggregating responses from combinations of the same LLMs. Results: We find that aggregating responses from multiple, various LLMs leads to more accurate differential diagnoses (average accuracy for 3 LLMs: $75.3\%\pm 1.6pp$) compared to the differential diagnoses produced by single LLMs (average accuracy for single LLMs: $59.0\%\pm 6.1pp$). Discussion: The use of collective intelligence methods to synthesize differential diagnoses combining the responses of different LLMs achieves two of the necessary steps towards advancing acceptance of LLMs as a diagnostic support tool: (1) demonstrate high diagnostic accuracy and (2) eliminate dependence on a single commercial vendor.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
