ProMRVL-CAD：面向计算机辅助诊断的具有多轮视觉-语言交互的主动对话系统
ProMRVL-CAD: Proactive Dialogue System with Multi-Round Vision-Language Interactions for Computer-Aided Diagnosis
AI总结：
针对LLM生成的医患交互与实际咨询存在差距的问题，提出ProMRVL-CAD系统，通过知识图谱集成、双生成器设计提升报告质量，在两个数据集上表现更优且低图像质量下稳健，还构建了合成对话数据集。
摘要：
大型语言模型（LLMs）的最新进展展示了卓越的理解能力，在各种视觉-语言任务上取得了显著突破。然而，LLMs在生成可靠医学诊断报告中的应用仍处于早期阶段。目前，医学LLMs通常采用被动交互模式，医生响应患者查询时很少或不参与医学图像分析。相比之下，一些聊天机器人仅基于视觉输入响应预定义查询，缺乏交互式对话或对病史的考虑。因此，LLM生成的患者-聊天机器人交互与实际医患咨询中的交互之间存在差距。为弥合这一差距，我们开发了一个基于LLM的对话系统，即面向计算机辅助诊断的主动多轮视觉-语言交互系统（ProMRVL-CAD），以生成患者友好的疾病诊断报告。所提出的ProMRVL-CAD系统通过将知识图谱集成到推荐系统中，允许主动对话为患者提供持续可靠的医疗服务。具体而言，我们设计了两个生成器：主动问题生成器（Pro-Q Gen），用于生成引导诊断过程的主动问题；多视觉患者-文本诊断报告生成器（MVP-DR Gen），用于生成高质量诊断报告。在两个真实世界公开可用数据集MIMIC-CXR和IU-Xray上进行评估，我们的模型在生成医学报告方面质量更优。我们进一步证明，ProMRVL在低图像质量场景下表现稳健。此外，我们创建了一个合成医学对话数据集，模拟患者与医生之间的主动诊断交互，作为训练LLM的宝贵资源。
英文摘要：
Recent advancements in large language models (LLMs) have demonstrated extraordinary comprehension capabilities with remarkable breakthroughs on various vision-language tasks. However, the application of LLMs in generating reliable medical diagnostic reports remains in the early stages. Currently, medical LLMs typically feature a passive interaction model where doctors respond to patient queries with little or no involvement in analyzing medical images. In contrast, some ChatBots simply respond to predefined queries based on visual inputs, lacking interactive dialogue or consideration of medical history. As such, there is a gap between LLM-generated patient-ChatBot interactions and those occurring in actual patient-doctor consultations. To bridge this gap, we develop an LLM-based dialogue system, namely proactive multi-round vision-language interactions for computer-aided diagnosis (ProMRVL-CAD), to generate patient-friendly disease diagnostic reports. The proposed ProMRVL-CAD system allows proactive dialogue to provide patients with constant and reliable medical access via an integration of knowledge graph into a recommendation system. Specifically, we devise two generators: a Proactive Question Generator (Pro-Q Gen) to generate proactive questions that guide the diagnostic procedure and a Multi-Vision Patient-Text Diagnostic Report Generator (MVP-DR Gen) to produce high-quality diagnostic reports. Evaluating two real-world publicly available datasets, MIMIC-CXR and IU-Xray, our model has better quality in generating medical reports. We further demonstrate the performance of ProMRVL achieves robust under the scenarios with low image quality. Moreover, we have created a synthetic medical dialogue dataset that simulates proactive diagnostic interactions between patients and doctors, serving as a valuable resource for training LLM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
