Anatomy-VLM：用于医学解读的细粒度视觉语言模型
Anatomy-VLM: A Fine-grained Vision-Language Model for Medical Interpretation
AI总结：
针对放射学解读因成像异质性和主流VLMs忽视细粒度细节的问题，本文受临床医生工作流程启发提出Anatomy-VLM，通过定位解剖特征、融合结构化知识和对齐多尺度信息，在多数据集及下游任务中表现优异，还支持零样本解剖解读。
摘要：
由于成像异质性，从放射学中准确解读疾病仍具挑战性。实现专家级诊断决策需要将细微图像特征与临床知识相结合。然而，主流视觉语言模型（VLMs）将图像视为整体实体，忽视了对疾病诊断至关重要的细粒度图像细节。临床医生利用其先验医学知识分析图像，并将解剖结构识别为重要感兴趣区域（ROIs）。受这种以人为中心的工作流程启发，我们引入Anatomy-VLM，一种整合多尺度信息的细粒度视觉语言模型。首先，我们设计模型编码器从整个医学图像中定位关键解剖特征。其次，这些区域通过结构化知识得到丰富，以实现上下文感知解读。最后，模型编码器对齐多尺度医学信息，生成临床可解释的疾病预测。Anatomy-VLM在分布内和分布外数据集上均取得优异性能。我们还在下游图像分割任务上验证了Anatomy-VLM的性能，表明其细粒度对齐捕获了解剖和病理相关知识。此外，Anatomy-VLM的编码器支持零样本解剖层面解读，展现出强大的专家级临床解读能力。
英文摘要：
Accurate disease interpretation from radiology remains challenging due to imaging heterogeneity. Achieving expert-level diagnostic decisions requires integration of subtle image features with clinical knowledge. Yet major vision-language models (VLMs) treat images as holistic entities and overlook fine-grained image details that are vital for disease diagnosis. Clinicians analyze images by utilizing their prior medical knowledge and identify anatomical structures as important region of interests (ROIs). Inspired from this human-centric workflow, we introduce Anatomy-VLM, a fine-grained, vision-language model that incorporates multi-scale information. First, we design a model encoder to localize key anatomical features from entire medical images. Second, these regions are enriched with structured knowledge for contextually-aware interpretation. Finally, the model encoder aligns multi-scale medical information to generate clinically-interpretable disease prediction. Anatomy-VLM achieves outstanding performance on both in- and out-of-distribution datasets. We also validate the performance of Anatomy-VLM on downstream image segmentation tasks, suggesting that its fine-grained alignment captures anatomical and pathology-related knowledge. Furthermore, the Anatomy-VLM's encoder facilitates zero-shot anatomy-wise interpretation, providing its strong expert-level clinical interpretation capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
