单细胞和空间转录组学基础模型的统一基准测试揭示了上下文相关的泛化能力
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation
浏览论文内容
中文总结 AI 辅助
研究单细胞和空间转录组学基础模型通用性，用统一框架对六个模型进行基准测试，评估多项任务，发现模型性能有条件性，无模型主导，排名随多种因素变化，为模型选择提供指导并提出评判模型新依据。
中文摘要 AI 辅助
单细胞和空间基础模型有望提供可转移的生物学表示，但它们在不同模态、生物领域和分析任务中的通用性仍 largely 未得到测试。我们使用一个统一框架对六个代表性模型进行基准测试，该框架涵盖 scRNA-seq、空间转录组学和 Perturb-seq。我们评估了零样本和持续预训练聚类、监督注释、标记基因一致性和扰动预测。模型性能具有很强的条件性，没有一个模型在所有任务中占主导地位，排名会随模态、预处理、token 化、生物学先验、域转移和度量选择而变化。该基准为模型选择提供了实用指导，并认为未来模型应通过生物学泛化、可解释性和基于扰动的有效性来评判，而不仅仅是规模或排行榜性能。
英文摘要
Single-cell and spatial foundation models promise transferable biological representations, yet their generality remains largely untested across modalities, biological domains and analytical tasks. We benchmarked six representative models, Nicheformer, CellPLM, scGPT-spatial, GenePT, scELMo and Novae, using a harmonised framework spanning scRNA-seq, spatial transcriptomics and Perturb-seq. We evaluated zero-shot and continually pretrained clustering, supervised annotation, marker-gene concordance and perturbation prediction. Model performance was strongly conditional: expression-trained cell-level transformers best resolved many cell-identity tasks, spatial and graph-aware models better preserved tissue architecture, and language-derived gene embeddings were competitive for selected perturbation-response metrics. No model dominated across tasks, and rankings shifted with modality, preprocessing, tokenisation, biological prior, domain shift and metric choice. This benchmark provides practical guidance for model selection and argues that future models should be judged by biological generalisation, interpretability and perturbation-grounded validity, not by scale or leaderboard performance alone.
发表机构
- BioMedical Machine Learning Laboratory, School of Biomedical Engineering, UNSW Sydney（新南威尔士大学生物医学工程学院生物医学机器学习实验室）
- School of Biotechnology and Biomolecular Sciences, UNSW Sydney（新南威尔士大学生物技术与分子科学学院）
- School of Computing, Macquarie University（麦考瑞大学计算机学院）
- Strands（23Strands）
- Shenzhen Key Laboratory for High Performance Data Mining, Shenzhen Institute of Advanced Technology（深圳先进技术研究院高性能数据挖掘重点实验室）
- School of Biomedical Engineering, UNSW Sydney（新南威尔士大学生物医学院）
- Center of Excellence in Precision Medicine and Digital Health, Faculty of Dentistry, Chulalongkorn University（朱拉隆功大学牙科学院精准医疗与数字健康卓越中心）
- UNSW AI Institute, University of New South Wales（新南威尔士大学人工智能研究所）
- Shanghai Institute of Immunology, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院上海免疫学研究所）
机构由 AI 辅助整理，请以论文原文为准。
