MicroCLIP：通过对比语言-图像预训练连接文本与显微镜图像
MicroCLIP: Connecting Text and Microscopy Images via Contrastive Language--Image Pretraining
AI总结：
本研究提出MicroCLIP视觉-语言模型，构建含250万对荧光显微镜图像-文本的MicroPool-2M数据集，通过三级图像-文本关联预训练，在显微镜图像零样本分类与检索中性能领先，可泛化至未见过的生物学概念，有望成为荧光显微镜基础模型。
摘要：
理解荧光显微镜图像是实现生物学研究自动化的基础。然而，现有方法通常从闭集数据中学习预定义概念，限制了其在不同任务和生物学场景间的迁移能力。本研究中，我们提出MicroCLIP，这是一种视觉-语言模型，可借助自然语言监督学习通用的细胞及亚细胞表征，用于开集图像分类与检索。为预训练MicroCLIP，我们构建了MicroPool-2M数据集，该数据集包含约250万对荧光显微镜图像-文本配对，涵盖多样的细胞结构、细胞系及成像条件。将显微镜图像与自然语言中的生物学概念对齐颇具挑战，因为多数图像缺乏描述性标题和生物学知识标注。因此，我们分三个层级逐步建立图像-文本关联，为MicroCLIP提供更丰富的语义监督：（1）类别级对齐，将图像与亚细胞结构-细胞系组合定义的标准化标签关联；（2）知识级对齐，整合源自这些标签的生物学先验，以捕捉跨物种和细胞类型的关系；（3）细节级对齐，利用多模态大语言模型大规模地为图像-文本配对重新生成标题并进行验证，强调细粒度视觉特征。大量实验表明，MicroCLIP在显微镜图像的零样本分类和文本-图像检索中取得了领先性能，可从数千个候选中检索出具有特定细胞特征的图像，且无需额外监督即可泛化至未见过的生物学概念。这些结果凸显了MicroCLIP作为荧光显微镜基础模型的潜力。
英文摘要：
Understanding fluorescence microscopy images is fundamental to automating biological research. However, existing methods typically learn predefined concepts from closed-set data, limiting transfer across tasks and biological contexts. In this work, we introduce MicroCLIP, a vision-language model that learns generalizable cellular and subcellular representations from natural language supervision for open-set image classification and retrieval. To pretrain MicroCLIP, we construct MicroPool-2M, a dataset of approximately 2.5 million fluorescence microscopy image-text pairs spanning diverse cellular structures, cell lines, and imaging conditions. Aligning microscopy images with biological concepts in natural language is challenging because most images lack descriptive captions and biological knowledge annotations. We therefore progressively establish image-text associations at three levels to provide MicroCLIP with richer semantic supervision: (1) category-level alignment links images to standardized labels defined by subcellular structure-cell-line combinations; (2) knowledge-level alignment incorporates biological priors derived from these labels to capture relationships across species and cell types; and (3) detail-level alignment uses a multimodal large language model to recaption and verify image-text pairs at scale, emphasizing fine-grained visual characteristics. Extensive experiments demonstrate that MicroCLIP achieves leading performance in zero-shot classification and text-image retrieval on microscopy images. It retrieves images with specific cellular characteristics from thousands of candidates and generalizes to unseen biological concepts without additional supervision. These results highlight MicroCLIP's potential as a foundation model for fluorescence microscopy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
