重新审视CLIP：使用领域特定基础模型高效对齐3D MRI与表格数据
Revisiting CLIP: Efficient Alignment of 3D MRI and Tabular Data using Domain-Specific Foundation Models
AI总结：
本文通过训练领域特定的3D基础模型，仅用62个MRI扫描实现3D MRI与表格数据的CLIP式对齐，采用嵌入累积策略稳定训练，并在零样本分类中验证了有效性。
摘要：
多模态模型需要对齐的共享嵌入空间。然而，常见的基于CLIP的方法需要大量样本，并且不能原生支持3D或表格数据，而这两者在医学领域都至关重要。为了解决这些问题，我们重新审视CLIP风格的对齐，通过训练一个领域特定的3D基础模型作为图像编码器，并证明仅使用62个MRI扫描即可实现模态对齐。我们的方法得益于一种简单的嵌入累积策略，该策略是3D训练所必需的，它跨批次扩展负样本对的数量以稳定训练。我们对各种设计选择进行了全面评估，包括主干网络和损失函数的选择，并在零样本分类和图像检索任务上评估了所提出的方法。虽然零样本图像检索仍然具有挑战性，但零样本分类结果表明，所提出的方法能够有意义地对齐3D MRI与表格数据的表示。
英文摘要：
Multi-modal models require aligned, shared embedding spaces. However, common CLIP-based approaches need large amounts of samples and do not natively support 3D or tabular data, both of which are crucial in the medical domain. To address these issues, we revisit CLIP-style alignment by training a domain-specific 3D foundation model as an image encoder and demonstrate that modality alignment is feasible with only 62 MRI scans. Our approach is enabled by a simple embedding accumulation strategy required for training in 3D, which scales the amount of negative pairs across batches in order to stabilize training. We perform a thorough evaluation of various design choices, including the choice of backbone and loss functions, and evaluate the proposed methodology on zero-shot classification and image-retrieval tasks. While zero-shot image-retrieval remains challenging, zero-shot classification results demonstrate that the proposed approach can meaningfully align the representations of 3D MRI with tabular data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
