GPT-4V在医学图像分析中多模态能力的系统性评估
A Systematic Evaluation of GPT-4V's Multimodal Capability for Medical Image Analysis
AI总结：
本研究系统评估GPT-4V的医学图像分析多模态能力，针对三类任务设计提示词并采用三种评估方式，发现其在报告生成和视觉问答上表现优异，但视觉定位能力待提升，且传统评估指标存在局限。
摘要：
本工作对GPT-4V（生成式预训练Transformer 4视觉版）的医学图像分析多模态能力开展评估，聚焦放射报告生成、医学视觉问答、医学视觉定位三项代表性任务。为完成评估，研究针对每项任务设计了一组提示词，以激发GPT-4V的对应能力，使其生成质量足够的输出。研究采用定量分析、人工评估、案例研究三种评估方式，实现深入且全面的评测。评估结果显示，GPT-4V在医学图像理解方面表现出色，能够生成高质量的放射报告，并有效解答与医学图像相关的问题。同时研究发现，其在医学视觉定位任务上的性能仍需大幅提升。此外，研究观察到定量分析的评估结果与人工评估的结果存在差异。这一差异表明，传统指标在评估GPT-4V这类大语言模型的性能时存在局限性，也凸显了开发新型自动定量分析指标的必要性。
英文摘要：
This work conducts an evaluation of GPT-4V's multimodal capability for medical image analysis, with a focus on three representative tasks of radiology report generation, medical visual question answering, and medical visual grounding. For the evaluation, a set of prompts is designed for each task to induce the corresponding capability of GPT-4V to produce sufficiently good outputs. Three evaluation ways including quantitative analysis, human evaluation, and case study are employed to achieve an in-depth and extensive evaluation. Our evaluation shows that GPT-4V excels in understanding medical images and is able to generate high-quality radiology reports and effectively answer questions about medical images. Meanwhile, it is found that its performance for medical visual grounding needs to be substantially improved. In addition, we observe the discrepancy between the evaluation outcome from quantitative analysis and that from human evaluation. This discrepancy suggests the limitations of conventional metrics in assessing the performance of large language models like GPT-4V and the necessity of developing new metrics for automatic quantitative analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
