使用大型视觉-语言模型进行多模态颈动脉风险分层：基准测试、微调与临床洞见
Multimodal Carotid Risk Stratification with Large Vision-Language Models: Benchmarking, Fine-Tuning, and Clinical Insights
AI总结：
本文评估开源LVLMs融合超声影像与临床表格数据进行颈动脉风险分层的能力，并通过LoRA微调LLaVa-NeXT-Vicuna显著提升卒中风险预测性能。
摘要：
颈动脉粥样硬化病的可靠风险评估仍是一项重大临床挑战，因为它需要以对临床医生透明且可解释的方式整合多样的临床和影像信息。本研究通过将超声成像（USI）与结构化临床、人口统计学、实验室及蛋白质生物标志物数据相结合，考察了最先进及近期大型视觉-语言模型（LVLMs）在多模态颈动脉斑块评估中的潜力。研究提出了一个通过访谈式问题序列模拟真实诊断场景的框架，比较了一系列开源LVLMs，包括通用模型和经过医学调优的模型。零样本实验表明，尽管这些模型非常强大，但并非所有LVLMs都能准确识别成像模态和解剖结构，而且它们在准确风险分类方面表现均较差。为解决这一局限，研究使用低秩适配（LoRA）使LLaVa-NeXT-Vicuna适配超声领域，从而显著改善了卒中风险分层。以文本形式整合多模态表格数据进一步提高了特异度和平衡准确率，与在同一数据集上训练的既往卷积神经网络（CNN）基线相比取得了具有竞争力的性能。我们的发现凸显了LVLMs在基于超声的心血管风险预测中的前景与局限，强调了多模态整合、模型校准和领域适配对于临床转化的重要性。
英文摘要：
Reliable risk assessment for carotid atheromatous disease remains a major clinical challenge, as it requires integrating diverse clinical and imaging information in a manner that is transparent and interpretable to clinicians. This study investigates the potential of state-of-the-art and recent large vision-language models (LVLMs) for multimodal carotid plaque assessment by integrating ultrasound imaging (USI) with structured clinical, demographic, laboratory, and protein biomarker data. A framework that simulates realistic diagnostic scenarios through interview-style question sequences is proposed, comparing a range of open-source LVLMs, including both general-purpose and medically tuned models. Zero-shot experiments reveal that even if they are very powerful, not all LVLMs can accurately identify imaging modality and anatomy, while all of them perform poorly in accurate risk classification. To address this limitation, LLaVa-NeXT-Vicuna is adapted to the ultrasound domain using low-rank adaptation (LoRA), resulting in substantial improvements in stroke risk stratification. The integration of multimodal tabular data in the form of text further enhances specificity and balanced accuracy, yielding competitive performance compared to prior convolutional neural network (CNN) baselines trained on the same dataset. Our findings highlight both the promise and limitations of LVLMs in ultrasound-based cardiovascular risk prediction, underscoring the importance of multimodal integration, model calibration, and domain adaptation for clinical translation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
