TEDDY: A Family Of Foundation Models For Understanding Single Cell Biology
BCG AI Science Institute, Boston, USA(BCG人工智能科学研究所,波士顿,美国);MSD (UK) Limited, London, UK(默沙东(英国)有限公司,伦敦,英国)
Alexis Chevalier, Soumya Ghosh, Urvi Awasthi, James Watkins, Julia Bieniewska, Nichita Mitrea, Olga Kotova, Kirill Shkura, Andrew Noble, Michael Steinbaugh, Vij… 展开作者
Alexis Chevalier, Soumya Ghosh, Urvi Awasthi, James Watkins, Julia Bieniewska, Nichita Mitrea, Olga Kotova, Kirill Shkura, Andrew Noble, Michael Steinbaugh, Vijay Sadashivaiah, George Dasoulas, Julien Delile, Christoph Meier, Leonid Zhukov, Iya Khalil, Srayanta Mukherjee, Judith Mueller
备注:
ICML 2025 Generative AI and Biology (GenBio) Workshop
Understanding the biological mechanisms of disease is crucial for medicine, and in particular, for drug discovery. AI-powered analysis of genome-scale biological data holds great potential in this regard. The increasing availability of single-cell RNA sequencing data has enabled the development of large foundation models for disease biology. However, existing foundation models only modestly improve over task-specific models in downstream applications. Here, we explored two avenues for improving single-cell foundation models. First, we scaled the pre-training data to a diverse collection of 116 million cells, which is larger than those used by previous models. Second, we leveraged the availability of large-scale biological annotations as a form of supervision during pre-training. We trained the \model family of models comprising six transformer-based state-of-the-art single-cell foundation models with 70 million, 160 million, and 400 million parameters. We vetted our models on several downstream evaluation tasks, including identifying the underlying disease state of held-out donors not seen during training, distinguishing between diseased and healthy cells for disease conditions and donors not seen during training, and probing the learned representations for known biology. Our models showed substantial improvement over existing works, and scaling experiments showed that performance improved predictably with both data volume and parameter count.
Character-level Tokenizations as Powerful Inductive Biases for RNA Foundational Models
Nostrum Biodiscovery(诺斯特伦生物发现公司);Institute for Research in Biomedicine (IRB Barcelona), The Barcelona Institute of Science and Technology(巴塞罗那科学技术研究所生物医药研究所(IRB巴塞罗那));Barcelona Supercomputing Center(巴塞罗那超级计算中心)
Arizona State University(亚利桑那州立大学);Purdue University(普渡大学);University of British Columbia(不列颠哥伦比亚大学);Stanford University(斯坦福大学);The Ohio State University(俄亥俄州立大学);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院);Virginia Polytechnic Institute(弗吉尼亚多项技术学院);University of Michigan(密歇根大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Nanjing University(南京大学)
The CausalBench challenge: A machine learning contest for gene network inference from single-cell perturbation data
Poznan University of Technology(波兹南理工大学);Lomonosov Moscow State University(莫斯科国立罗蒙诺索夫大学);Columbia University(哥伦比亚大学);Ryvu Therapeutics;Harvard Medical School(哈佛医学院);University of Oxford(牛津大学)
Toward AI-Driven Digital Organism: Multiscale Foundation Models for Predicting, Simulating and Programming Biology at All Levels
GenBio AI;Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Weizmann Institute of Science(魏茨曼科学研究所);Carnegie Mellon University(卡内基梅隆大学)
BioArc: Discovering Optimal Neural Architectures for Biological Foundation Models
Department of Computer Science, Virginia Tech, Blacksburg, VA, USA(弗吉尼亚理工学院计算机科学系);Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA, USA(弗吉尼亚理工学院电气与计算机工程系);Department of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA(卡内基梅隆大学计算机科学系);Department of Biomedical Data Science, Stanford University, Stanford, CA, USA(斯坦福大学生物医学数据科学系)
A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language
Renmin University of China(中国人民大学);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences(中国科学院大学);The Chinese University of Hong Kong(香港中文大学)
Towards Applying Large Language Models to Complement Single-Cell Foundation Models
University of Toronto(多伦多大学);Vector Institute(向量研究所);Peter Munk Cardiac Centre, University Health Network(彼得·穆恩心脏中心,大学健康网络);AI Hub, University Health Network(人工智能中心,大学健康网络);Princess Margaret Cancer Centre, University Health Network(玛格丽特公主癌症中心,大学健康网络);Lunenfeld-Tanenbaum Research Institute, Sinai Health System(伦纳尔德-坦恩堡研究所,辛纳医院系统)
SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Department of Computer Science and Tecnology, Tsinghua University(清华大学计算机科学与技术系);Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院);School of Mathematical Sciences, Peking University(北京大学数学科学学院);PharMolix Inc.(PharMolix公司)
VitaGraph: Building a Knowledge Graph for Biologically Relevant Learning Tasks
DIAG, Sapienza University of Rome(Sapienza罗马大学DIAG部门);IAC, CNR Consiglio Nazionale delle Ricerche(CNR国家研究委员会IAC部门);Department of Biochemical Sciences(生物化学系);Sapienza University of Rome(Sapienza罗马大学);Department of Life Science Informatics and Data Science(生命科学信息学与数据科学系);University of Bonn(波恩大学);Lamarr Institute(拉玛尔研究所);DIET, Sapienza University of Rome(Sapienza罗马大学DIET部门)
Blue Marble Space Institute of Science(蓝色大理石空间科学研究所);NASA Ames Research Center(美国国家航空航天局艾姆斯研究中心);Rutgers New Jersey Medical School(罗格斯新泽西医学院);KBR(KBR公司);University of Texas, San Antonio(德克萨斯大学圣安东尼奥分校);UT Health Sciences, San Antonio(德克萨斯大学圣安东尼奥健康科学中心);Lawrence Berkeley National Lab(劳伦斯伯克利国家实验室);DOE Agile BioFoundry(美国能源部敏捷生物铸造厂);Joint BioEnergy Institute(联合生物能源研究所);Weill Cornell Medicine(威尔康奈尔医学院);Bay Area Environmental Research Institute(湾区环境研究所);Universities Space Research Association(大学空间研究协会);University of North Florida(北佛罗里达大学);Mayo Clinic(梅奥诊所);Baylor College of Medicine(贝勒医学院);University of California San Francisco(加利福尼亚大学旧金山分校);The University of Wisconsin - Madison(威斯康星大学麦迪逊分校);Georgia Institute of Technology(佐治亚理工学院);Hasso Plattner Institute for Digital Health at Mount Sinai(西奈山哈索·普拉特纳数字健康研究所);Icahn School of Medicine at Mount Sinai(西奈山伊坎医学院)