Swin Transformer 的自监督模态无关预训练
Self-Supervised Modality-Agnostic Pre-Training of Swin Transformers
AI总结：
本文提出 SwinFUSE，通过融合 CT 与 MRI 的自监督预训练及领域不变性模块增强 Swin Transformer，以轻微同域性能代价换取分布外医学图像分割最高 27% 的提升。
摘要：
无监督预训练已成为一种变革性范式，在多个领域展现出显著进展。然而，其易受领域漂移影响的问题——即预训练数据分布与微调时不同——构成了重大障碍。为解决该问题，我们增强 Swin Transformer，使其能够从不同医学成像模态中学习，从而提升下游性能。我们的模型名为 SwinFUSE（Swin Multi-Modal Fusion for UnSupervised Enhancement，用于无监督增强的 Swin 多模态融合），具有三个关键优势：（i）它在预训练期间同时从计算机断层扫描（CT）和磁共振图像（MRI）中学习，产生互补的特征表示；（ii）领域不变性模块（DIM）可有效突出显著输入区域，增强适应性；（iii）表现出显著的泛化能力，超越其最初预训练任务的限制。我们在两个公开可用的 3D 分割数据集上的实验表明，与单模态模型相比，性能仅有 1–2% 的轻微折衷，但在分布外模态上的性能显著提升最高达 27%。这一实质性改进凸显了所提方法的实际相关性和现实适用性。代码可在 https://github.com/devalab/SwinFUSE 获取。
英文摘要：
Unsupervised pre-training has emerged as a transformative paradigm, displaying remarkable advancements in various domains. However, the susceptibility to domain shift, where pre-training data distribution differs from fine-tuning, poses a significant obstacle. To address this, we augment the Swin Transformer to learn from different medical imaging modalities, enhancing downstream performance. Our model, dubbed SwinFUSE (Swin Multi-Modal Fusion for UnSupervised Enhancement), offers three key advantages: (i) it learns from both Computed Tomography (CT) and Magnetic Resonance Images (MRI) during pre-training, resulting in complementary feature representations; (ii) a domain-invariance module (DIM) that effectively highlights salient input regions, enhancing adaptability; (iii) exhibits remarkable generalizability, surpassing the confines of tasks it was initially pre-trained on. Our experiments on two publicly available 3D segmentation datasets show a modest 1-2% performance trade-off compared to single-modality models, yet significant out-performance of up to 27% on out-of-distribution modality. This substantial improvement underscores our proposed approach's practical relevance and real-world applicability. Code is available at: https://github.com/devalab/SwinFUSE
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
