CytoCLIP:利用对比语言图像预训练学习发育人类大脑的细胞架构特征
CytoCLIP: Learning Cytoarchitectural Characteristics in Developing Human Brain Using Contrastive Language Image Pre-Training
AI总结:
CytoCLIP通过对比语言图像预训练框架学习人类大脑细胞架构特征,通过低分辨率和高分辨率模型变体实现区域分类和跨模态检索,实验表明其在整体区域和高分辨率图像分类中表现优异。
摘要:
人类大脑不同区域的功能与其独特的细胞架构密切相关,细胞架构由细胞的空间排列和形态决定。通过细胞架构识别脑区可进行多种脑科学分析。然而,手动在脑组织切片中界定这些区域耗时且需要专业知识。为此,我们提出CytoCLIP,一种基于预训练对比语言-图像预训练(CLIP)框架的视觉-语言模型集,用于学习脑细胞架构的联合视觉-文本表示。CytoCLIP包含两种模型变体:一种使用低分辨率全区域图像训练以理解区域的整体细胞架构模式,另一种使用高分辨率图像切片训练以获得详细的细胞级表示。训练数据集来自不同妊娠周数的胎儿大脑的尼氏染色组织切片,包括86个不同区域用于低分辨率图像和379个脑区域用于高分辨率切片。我们通过区域分类和跨模态检索任务评估模型对细胞架构的理解和泛化能力。在不同数据设置下进行了多项实验,包括不同年龄和切片平面的样本数据。实验结果表明,CytoCLIP优于现有方法,实现了整体区域分类的加权F1得分为0.87,高分辨率图像切片分类得分为0.91。
英文摘要:
The functions of different regions of the human brain are closely linked to their distinct cytoarchitecture, which is defined by the spatial arrangement and morphology of the cells. Identifying brain regions by their cytoarchitecture enables various scientific analyses of the brain. However, delineating these areas manually in brain histological sections is time-consuming and requires specialized knowledge. An automated approach is necessary to minimize the effort needed from human experts. To address this, we propose CytoCLIP, a suite of vision-language models derived from pre-trained Contrastive Language-Image Pre-Training (CLIP) frameworks to learn joint visual-text representations of brain cytoarchitecture. CytoCLIP comprises two model variants: one is trained using low-resolution whole-region images to understand the overall cytoarchitectural pattern of an area, and the other is trained on high-resolution image tiles for detailed cellular-level representation. The training dataset is created from NISSL-stained histological sections of developing fetal brains of different gestational weeks. It includes 86 distinct regions for low-resolution images and 379 brain regions for high-resolution tiles. We evaluate the model's understanding of the cytoarchitecture and generalization ability using region classification and cross-modal retrieval tasks. Multiple experiments are performed under various data setups, including data from samples of different ages and sectioning planes. Experimental results demonstrate that CytoCLIP outperforms existing methods. It achieves a weighted F1 score of 0.87 for whole-region classification and 0.91 for high-resolution image tile classification.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。