医学影像诊断中多模态AI模型的综合评估：从数据增强到基于偏好的比较
Comprehensive Evaluation of Multimodal AI Models in Medical Imaging Diagnosis: From Data Augmentation to Preference-Based Comparison
AI总结：
本研究提出医学影像多模态AI评估框架，通过数据扩增扩充病例库，基于偏好对比评估发现Llama 3.2-90B在多数病例中优于人类诊断，专用视觉模型表现稍逊，验证了大模型在相关诊断任务的潜力。
摘要：
本研究提出了一套面向医学影像诊断领域多模态模型的评估框架。我们构建了包含数据预处理、模型推理与基于偏好的评估的流水线，通过受控增强将初始的500个临床病例扩展至3000个。该方法结合医学影像与临床观察生成诊断评估，采用Claude 3.5 Sonnet以医生出具的诊断结果为基准开展独立评估。结果显示不同模型表现存在差异：Llama 3.2-90B在85.27%的病例中表现优于人类诊断；相比之下，BLIP2、Llava等专用视觉模型的偏好占比分别为41.36%和46.77%。该框架凸显了大型多模态模型在特定任务中超越人类诊断水平的潜力。
英文摘要：
This study introduces an evaluation framework for multimodal models in medical imaging diagnostics. We developed a pipeline incorporating data preprocessing, model inference, and preference-based evaluation, expanding an initial set of 500 clinical cases to 3,000 through controlled augmentation. Our method combined medical images with clinical observations to generate assessments, using Claude 3.5 Sonnet for independent evaluation against physician-authored diagnoses. The results indicated varying performance across models, with Llama 3.2-90B outperforming human diagnoses in 85.27% of cases. In contrast, specialized vision models like BLIP2 and Llava showed preferences in 41.36% and 46.77% of cases, respectively. This framework highlights the potential of large multimodal models to outperform human diagnostics in certain tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
