医学表征学习中视觉-语言对比方法的基准评测
Benchmarking Vision-Language Contrastive Methods for Medical Representation Learning
AI总结：
本文通过统一设置基准评测八种医学视觉-语言对比学习方法，发现通用表征可迁移、多模态训练无需额外单模态训练，且细粒度特征更有益。
摘要：
我们对医学领域中用于学习多模态表征的对比框架进行了全面的基准评测。通过这项研究，我们旨在回答以下研究问题：(i) 通用领域表征向医学领域的可迁移性如何？(ii) 多模态对比训练是否足够，还是同样受益于单模态训练？(iii) 特征粒度对多模态医学表征学习有效性的影响是什么？为了回答这些问题，我们在相同的训练设置下研究了八种对比学习方法，并在来自四个数据集的280万对图像-文本对上训练它们，在25个下游任务上评估它们，包括分类（零样本和线性探测）、图像到文本和文本到图像检索，以及视觉问答。我们的发现表明第一个问题的答案是肯定的，第二个问题的答案是否定的，并且学习细粒度特征是有益的。最后，我们公开了我们的代码。
英文摘要：
We perform a comprehensive benchmarking of contrastive frameworks for learning multimodal representations in the medical domain. Through this study, we aim to answer the following research questions: (i) How transferable are general-domain representations to the medical domain? (ii) Is multimodal contrastive training sufficient, or does it benefit from unimodal training as well? (iii) What is the impact of feature granularity on the effectiveness of multimodal medical representation learning? To answer these questions, we investigate eight contrastive learning approaches under identical training setups, and train them on 2.8 million image-text pairs from four datasets, and evaluate them on 25 downstream tasks, including classification (zero-shot and linear probing), image-to-text and text-to-image retrieval, and visual question-answering. Our findings suggest a positive answer to the first question, a negative answer to the second question, and the benefit of learning fine-grained features. Finally, we make our code publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
