UMass-BioNLP 在 MEDIQA-M3G 2024 上的表现：DermPrompt——使用 GPT-4V 进行皮肤科诊断的提示工程系统探索
UMass-BioNLP at MEDIQA-M3G 2024: DermPrompt -- A Systematic Exploration of Prompt Engineering with GPT-4V for Dermatological Diagnosis
AI总结：
该研究在 MEDIQA-M3G 2024 中提出 DermPrompt，基于 GPT-4V、检索-重排序、CoT 与多智能体会话框架提升皮肤科多模态诊断。
摘要：
本文介绍了我们团队参与 MEDIQA-ClinicalNLP2024 共享任务 B 的工作。我们提出了一种诊断临床皮肤科病例的新方法，通过集成大型多模态模型，特别是在检索器和重排序器框架下利用 GPT-4V 的能力。我们的研究表明，当 GPT-4V 用作检索智能体时，借助皮肤科图像和简要患者病史，它能够在 85% 的情况下准确检索到正确的皮肤病症。此外，我们通过实验证明，朴素思维链（CoT）在检索任务中表现良好，而准确的皮肤科诊断则需要基于医学指南的 CoT。进一步地，我们提出了多智能体会话（MAC）框架，并展示了其相对于最佳 CoT 策略的优越性能和潜力。实验表明，将朴素 CoT 用于检索、将多智能体会话用于基于批评的诊断，GPT-4V 可实现皮肤科病症的早期准确诊断。本工作的意义延伸至改进诊断工作流程、支持皮肤科教育，以及通过提供可扩展、易获取且准确的诊断工具来改善患者护理。
英文摘要：
This paper presents our team's participation in the MEDIQA-ClinicalNLP2024 shared task B. We present a novel approach to diagnosing clinical dermatology cases by integrating large multimodal models, specifically leveraging the capabilities of GPT-4V under a retriever and a re-ranker framework. Our investigation reveals that GPT-4V, when used as a retrieval agent, can accurately retrieve the correct skin condition 85% of the time using dermatological images and brief patient histories. Additionally, we empirically show that Naive Chain-of-Thought (CoT) works well for retrieval while Medical Guidelines Grounded CoT is required for accurate dermatological diagnosis. Further, we introduce a Multi-Agent Conversation (MAC) framework and show its superior performance and potential over the best CoT strategy. The experiments suggest that using naive CoT for retrieval and multi-agent conversation for critique-based diagnosis, GPT-4V can lead to an early and accurate diagnosis of dermatological conditions. The implications of this work extend to improving diagnostic workflows, supporting dermatological education, and enhancing patient care by providing a scalable, accessible, and accurate diagnostic tool.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
