开放世界知识辅助的单细胞基础模型与鲁棒跨模态细胞-语言预训练
Open World Knowledge Aided Single-Cell Foundation Model with Robust Cross-Modal Cell-Language Pre-training
AI总结：
OKR-CELL通过开放世界知识和鲁棒跨模态预训练，提升单细胞基础模型在细胞-语言跨模态任务中的表现。
摘要：
最近单细胞多组学，特别是RNA测序，的进步为细胞异质性和基因调控提供了深刻见解。虽然基于预训练语言模型（PLM）范式的单细胞基础模型显示出潜力，但它们仍然受限于缺乏深入的个体资料整合和忽视多模态数据中的噪声影响。为了解决这两个问题，我们提出了开放世界语言知识辅助的鲁棒单细胞基础模型（OKR-CELL）。该模型基于跨模态细胞-语言预训练框架，包含两个关键创新：（1）利用基于大语言模型（LLMs）的工作流，结合检索增强生成（RAG）来利用开放世界知识丰富细胞文本描述；（2）设计了跨模态鲁棒对齐（CRA）目标，该目标结合了样本可靠性评估、课程学习和耦合动量对比学习，以增强模型对噪声数据的抗性。在预训练3200万细胞-文本对后，OKR-CELL在6个评估任务上取得了最先进的结果。除了标准基准，如细胞聚类、细胞类型注释、批次效应校正和少量样本注释外，该模型还在更广泛的多模态应用中表现出色，包括零样本细胞类型注释和双向细胞-文本检索。
英文摘要：
Recent advancements in single-cell multi-omics, particularly RNA-seq, have provided profound insights into cellular heterogeneity and gene regulation. While pre-trained language model (PLM) paradigm based single-cell foundation models have shown promise, they remain constrained by insufficient integration of in-depth individual profiles and neglecting the influence of noise within multi-modal data. To address both issues, we propose an Open-world Language Knowledge-Aided Robust Single-Cell Foundation Model (OKR-CELL). It is built based on a cross-modal Cell-Language pre-training framework, which comprises two key innovations: (1) leveraging Large Language Models (LLMs) based workflow with retrieval-augmented generation (RAG) enriches cell textual descriptions using open-world knowledge; (2) devising a Cross-modal Robust Alignment (CRA) objective that incorporates sample reliability assessment, curriculum learning, and coupled momentum contrastive learning to strengthen the model's resistance to noisy data. After pretraining on 32M cell-text pairs, OKR-CELL obtains cutting-edge results across 6 evaluation tasks. Beyond standard benchmarks such as cell clustering, cell-type annotation, batch-effect correction, and few-shot annotation, the model also demonstrates superior performance in broader multi-modal applications, including zero-shot cell-type annotation and bidirectional cell-text retrieval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
