多模态掩码孪生网络改进胸部X光表示学习
Multi-modal Masked Siamese Network Improves Chest X-Ray Representation Learning
AI总结：
本文提出在掩码孪生网络自监督预训练中融入电子健康记录数据，通过在三个胸部X光数据集上的实验证明该方法能显著提升表示学习质量，优于现有自监督基线。
摘要：
针对医学图像的自监督学习方法在预训练阶段主要依赖成像模态。尽管此类方法取得了有前景的结果，但它们并未利用电子健康记录（EHR）中收集的相关患者或扫描信息。在此，我们提出在自监督预训练过程中通过掩码孪生网络（MSN）融入EHR数据，以提升胸部X光表示的质量。我们研究了三种类型的EHR数据，包括人口统计学信息、扫描元数据和住院信息。我们在三个公开可用的胸部X光数据集MIMIC-CXR、CheXpert和NIH-14上评估了我们的方法，使用两种视觉Transformer（ViT）骨干网络，具体为ViT-Tiny和ViT-Small。通过线性评估来评估表示质量时，我们提出的方法与原始MSN和当前最先进的自监督学习基线相比，展示了显著的改进。我们的工作凸显了EHR增强的自监督预训练在医学成像领域的潜力。代码公开可用：https://github.com/nyuad-cai/CXR-EHR-MSN
英文摘要：
Self-supervised learning methods for medical images primarily rely on the imaging modality during pretraining. While such approaches deliver promising results, they do not leverage associated patient or scan information collected within Electronic Health Records (EHR). Here, we propose to incorporate EHR data during self-supervised pretraining with a Masked Siamese Network (MSN) to enhance the quality of chest X-ray representations. We investigate three types of EHR data, including demographic, scan metadata, and inpatient stay information. We evaluate our approach on three publicly available chest X-ray datasets, MIMIC-CXR, CheXpert, and NIH-14, using two vision transformer (ViT) backbones, specifically ViT-Tiny and ViT-Small. In assessing the quality of the representations via linear evaluation, our proposed method demonstrates significant improvement compared to vanilla MSN and state-of-the-art self-supervised learning baselines. Our work highlights the potential of EHR-enhanced self-supervised pre-training for medical imaging. The code is publicly available at: https://github.com/nyuad-cai/CXR-EHR-MSN
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
