arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.03485cs.LGq-bio.QM
收藏

TEDDY:单细胞生物学理解的模型家族

TEDDY: A Family Of Foundation Models For Understanding Single Cell Biology

BCG AI Science Institute, Boston, USA(BCG人工智能科学研究所,波士顿,美国);MSD (UK) Limited, London, UK(默沙东(英国)有限公司,伦敦,英国)
Alexis Chevalier, Soumya Ghosh, Urvi Awasthi, James Watkins, Julia Bieniewska, Nichita Mitrea, Olga Kotova, Kirill Shkura, Andrew Noble, Michael Steinbaugh, Vij… 展开作者

Alexis Chevalier, Soumya Ghosh, Urvi Awasthi, James Watkins, Julia Bieniewska, Nichita Mitrea, Olga Kotova, Kirill Shkura, Andrew Noble, Michael Steinbaugh, Vijay Sadashivaiah, George Dasoulas, Julien Delile, Christoph Meier, Leonid Zhukov, Iya Khalil, Srayanta Mukherjee, Judith Mueller

AI总结:

本文提出TEDDY模型家族,通过扩大预训练数据和利用大规模生物注释提升单细胞基础模型性能,验证了模型在疾病状态识别和生物学探索中的有效性。

摘要:

理解疾病生物学机制对医学至关重要,尤其是药物发现。AI分析基因组级生物数据在其中具有巨大潜力。单细胞RNA测序数据的可用性促进了大规模基础模型的发展。然而,现有基础模型在下游应用中仅小幅改进。本文通过扩大预训练数据至1.16亿个细胞和利用大规模生物注释作为预训练监督,训练了包含六个基于transformer的先进单细胞基础模型的家族,参数量分别为7000万、1.6亿和4亿。模型在多个下游任务中表现出显著改进,包括识别训练未见的受试者疾病状态、区分训练未见的疾病细胞和健康细胞,以及探索已知生物学的表示。模型性能随数据量和参数量增加而可预测性提升。

英文摘要:

Understanding the biological mechanisms of disease is crucial for medicine, and in particular, for drug discovery. AI-powered analysis of genome-scale biological data holds great potential in this regard. The increasing availability of single-cell RNA sequencing data has enabled the development of large foundation models for disease biology. However, existing foundation models only modestly improve over task-specific models in downstream applications. Here, we explored two avenues for improving single-cell foundation models. First, we scaled the pre-training data to a diverse collection of 116 million cells, which is larger than those used by previous models. Second, we leveraged the availability of large-scale biological annotations as a form of supervision during pre-training. We trained the \model family of models comprising six transformer-based state-of-the-art single-cell foundation models with 70 million, 160 million, and 400 million parameters. We vetted our models on several downstream evaluation tasks, including identifying the underlying disease state of held-out donors not seen during training, distinguishing between diseased and healthy cells for disease conditions and donors not seen during training, and probing the learned representations for known biology. Our models showed substantial improvement over existing works, and scaling experiments showed that performance improved predictably with both data volume and parameter count.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑