评估大型语言模型在妇产科病史采集中的可行性与准确性
Evaluating the Feasibility and Accuracy of Large Language Models for Medical History-Taking in Obstetrics and Gynecology
AI总结：
本研究评估了ChatGPT-4o和4o-mini在不孕不育病史采集中的可行性与准确性，发现4o-mini在信息提取和完整性上表现更优，而4o在鉴别诊断上略胜，两者均具较高自动化可行性。
摘要：
在诊断前环境中，特别是在不孕不育等复杂且敏感的医疗领域，有效的医患沟通至关重要，但非常耗时，因此会导致临床工作流程效率低下。大型语言模型（LLMs）的最新进展为自动化对话式病史采集和提高诊断准确性提供了潜在的解决方案。本研究评估了LLMs在不孕不育病例中执行这些任务的可行性和性能。开发了一个AI驱动的对话系统，使用ChatGPT-4o和ChatGPT-4o-mini模拟医患互动。共处理了70个真实世界的不孕不育病例，生成了420份诊断病史。使用F1分数、鉴别诊断（DDs）准确率和不孕不育类型判断（ITJ）准确率来评估模型性能。ChatGPT-4o-mini在信息提取准确性上优于ChatGPT-4o（F1分数：0.9258 vs. 0.9029，p = 0.045，d = 0.244），并在病史采集完整性上表现出更高水平（97.58% vs. 77.11%），这表明ChatGPT-4o-mini在提取详细患者信息方面更为有效，这对于提高诊断准确性至关重要。相比之下，ChatGPT-4o在鉴别诊断准确率上略高（2.0524 vs. 2.0048，p > 0.05）。ChatGPT-4o-mini的ITJ准确率更高（0.6476 vs. 0.5905），但一致性较低（Cronbach's α = 0.562），表明分类可靠性存在变异性。两种模型在自动化不孕不育病史采集方面均表现出很强的可行性，其中ChatGPT-4o-mini在完整性和提取准确性上表现优异。在未来的研究中，必须优先考虑在临床环境中对准确性和可靠性进行专家验证、AI模型微调，以及使用包含混合不孕不育病例的更大数据集。
英文摘要：
Effective physician-patient communications in pre-diagnostic environments, and most specifically in complex and sensitive medical areas such as infertility, are critical but consume a lot of time and, therefore, cause clinic workflows to become inefficient. Recent advancements in Large Language Models (LLMs) offer a potential solution for automating conversational medical history-taking and improving diagnostic accuracy. This study evaluates the feasibility and performance of LLMs in those tasks for infertility cases. An AI-driven conversational system was developed to simulate physician-patient interactions with ChatGPT-4o and ChatGPT-4o-mini. A total of 70 real-world infertility cases were processed, generating 420 diagnostic histories. Model performance was assessed using F1 score, Differential Diagnosis (DDs) Accuracy, and Accuracy of Infertility Type Judgment (ITJ). ChatGPT-4o-mini outperformed ChatGPT-4o in information extraction accuracy (F1 score: 0.9258 vs. 0.9029, p = 0.045, d = 0.244) and demonstrated higher completeness in medical history-taking (97.58% vs. 77.11%), suggesting that ChatGPT-4o-mini is more effective in extracting detailed patient information, which is critical for improving diagnostic accuracy. In contrast, ChatGPT-4o performed slightly better in differential diagnosis accuracy (2.0524 vs. 2.0048, p > 0.05). ITJ accuracy was higher in ChatGPT-4o-mini (0.6476 vs. 0.5905) but with lower consistency (Cronbach's $α$ = 0.562), suggesting variability in classification reliability. Both models demonstrated strong feasibility in automating infertility history-taking, with ChatGPT-4o-mini excelling in completeness and extraction accuracy. In future studies, expert validation for accuracy and dependability in a clinical setting, AI model fine-tuning, and larger datasets with a mix of cases of infertility have to be prioritized.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
